The first way to understand if your RDS is idle can be to integrate with AWS CloudWatch metrics, gather the relevant information, analyze it and try to understand what is idle and what is in use.
A much simpler option would be to use Trusted Advisor:
If a DB instance has not had a connection for a prolonged period of time, you can delete the instance to reduce costs. If persistent storage is needed for data on the instance, you can use lower-cost options such as taking and retaining a DB snapshot.
Alert Criteria:
Yellow: An active DB instance has not had a connection in the last 7 days.
Simple, right?
Get familiarized with the trusted advisor service, as you don't have to reinvent the wheel for specific things it can offer, but be mindful of the suggestions, and understand what they mean to make sure the business decisions support the recommendation.