I might change how the tweets are counted in the following weeks.

Currently the program filters a few things, then goes through the actual tweet text and I am only counting new content with hashtags,

Meaning the new tweet needs to include the hashtag..

The way Twitter considers a tweet has included a hashtag is a bit looser, as in,

If you retweet with comment a tweet with hashtag, but don't include a hashtag in the comment it can still be considered as if the tweet included a hashtag,..

If you replied to a tweet with hashtag without writing the hashtag, it might still be considered as if you used the hashtag,..

This is a looser interpretation that I avoided in the past, and going through tweet text is not that taxing on the system, but I could just take the hashtag Twitter thinks are associated with the tweet without parsing the tweets.

I will still filter unrelated hashtags, and third party app tweets, and tweets with #ad, but I think this will streamline things even further than the current approach, as I am now storing tweets on the server temporarily,..

Until they are downloaded to the charts processing offline computer to generate the charts data.

I hope this was not too confusing for everyone.

The reason the tweets are stored on an offline computer is related to cost of server storage. Server storage would be extremely expensive if I had to store +2GB per day, or +1TB every month. It's an exponential cost, where buying one hard drive, I buy it once, and don't need to..

Pay for it every month to keep the storage.