Monitoring log files

I am attempting to monitor our log files for various Exception patterns, however I only want an alert to be raised if the Exceptions occur a specific number of times over a given period: for example, 20 times in a period of 15 minutes.

I started by creating a File monitor that looked for the contents of the Exception, however whenever the exception occurred, an alert was immediately raised showing the file as "unavailable". I am using Jboss 4.2.2 and I know that you can use the log4j to riase JMX Notifications using the org.jboss.monitor.services.JMXNotificationAppender or you can raise SNMP traps using "org.apache.log4j.ext.JoeSNMPTrapSender". I really don't know how to use these within Applications Manager.

What is the most effective way to achieve the Exception monitoring requirement?

Thank you!