Everywhere in Windows, backspace allows you to delete one character to the left of cursor at a time. However, in some parts of the OS, Ctrl+Backspace is also supported. This allows the entire word to the left of the cursor to be deleted. For example, the Run dialog (Win+R) or Windows 10's start menu search. Some areas this has not been implemented, such as while renaming a folder in explorer. In such cases, explorer interprets it as some kind of unicode character and will insert a square box instead. The only alternative is to press backspace multiple times, or to press ctrl+shift+left arrow to select the previous word then press backspace, or if only one word has been typed, press ctrl+a to select all the text then backspace to delete it.
PortableApps's menu has the second behaviour, whereby Ctrl+Backspace will not delete the word to the left of the cursor but instead insert a square box character. I would like to propose a modification to this behaviour, it will benefit keyboard centric users.