Who are the people who ask for my poll number when I leave the polling station?

These people are called tellers and are used by each of the main political parties to help with their election campaigns. They have no standing in electoral law and not connected with the official election process.

You don't have to give them your poll number if you don't want to.