I found few topics about filtering by category. The answer to all of them was basically that it is coming soon as a new feature to the front end.

However, I found that there is actually a good workaround, which can satisfy a large number of needs. It is basically using a "menu" with "menu items" for each "category". It is demonstrated in the video tutorials, which are published by the author an iCagenda fan (thank you Giusebos). However, initially, I didn't get it, because the video has no explanation (voice or subtitles), the Joomla interface in the demo is not in English, and the demo uses an old version of this excellent extension. So, it was hard for me to follow.

Finally, I figured it out, and thought that some other people might be in the same position. So, here is the workaround...

The idea is to create a menu, and an iCagenda menu item for each of the categories that you want to be used for filtering. Then, add this menu to each of the events' pages. Following is a step-by-step explanation.

0. This is step zero, because you probably have already done it. Create a "menu item of type" iCajenda, and make sure the "Filter by Category" field is empty. Let's say this menu item is called (Events.)

1. Create a menu. Let's call it "Filter by Category"

2. Create a menu item for each category , which you want to use for filtering. While creating the following menu items, make sure they belong to the menu, which you created above (by checking the "Menu Location".)

For example, let's create a "menu item of type" iCajenda and call it (Children Education). In the "iCajenda options" tab, use the "Filter by Category" to specify the categories, which you consider as Children Education. You might pick English, Arabic, Science & Math.

Click on Save. Then, on Save as Copy. Change the name of the new item menu, which was just copied, to be (Adults Classes), for example. Then, in the "iCagenda Options" tab, use the "filter by Category" to specify the categories, which you consider as Adults' Classes. You might pick Business, IT & Plumbing.

3. Create a module for the menu, which you created in step 1. This can be done by following the following steps.
Menus -> Manage -> Find the new menu in the list, and click on "Add a module for this menu type."
-> Choose an appropriate position for this menu, e.g. a sidebar position
-> Menu Assignment -> Make sure you choose all of the new menu items, which you created above, i.e. (Events), (Children Education) & (Adults Classes)

That's it!!! If you browse to (Events), you should see a menu at the side having categorisation capabilities.

If I missed a step or miss explained something, please do not hesitate to correct me (for the benefit of the community).