On the Sly

An informal idiom to do something secretly, quietly or in a hidden manner, often because it is forbidden, disapproved of or considered wrong. The phrase comes from the adjective sly, which entered English language around 12th century from Old Norse word slœgr meaning cunning, crafty, or cleverly skillful 

In Chapter 7 of Great Expectations we see Joe taking lessons in alphabets and numbers.  Joe wants Pip to keep this a secret from Mrs Gargery