I cannot stand google news any more, too much spam, clickbait and advertisement. So I decided to try to selfhost an RSS aggregator to make myself a news feed that I would be comfortable with. Being RSS such an "ancient" thing I thought there will be many mature systems, but I'm not sure that's the case..
As far as my investigation goes there are two main options out there** TT-RSS (tiny tiny RSS) and FreshRSS**. There seems to also be miniflux but it supposedly have very few features.
So I tried the both main ones and I ended up kind of disappointed, I hope that I'm missing something. My requirements are:
1-Have a nice interface, card view, phone friendly. Basically being able to look the same as google news looked. So both have a pretty dated interface. And terrible responsive UI for phones. I was kind of able to make a "card view" with TT-RSS but looked hideous and didn't really work on phone screen, also applying themes broke TT-RSS, this will be recurring theme but it looks like TT-RSS is constantly breaking a rolling release system makes it very unstable and many plugins, themes and third party apps don't work right now because some new update broke everything. So native theming wasn't going to be a thing, so I tried third party apps. I found many that worked with FreshRSS and settled on Feedme, it looked exactly as I wanted, great. One point for FreshRSS. Feedme was supposedly compatible with TTRSS but I could not login, I have the suspicion that one update broke integration. I'm not even try to attempt to ask in their forums as I see that some time ago somebody asked the same question and got banned from their forums.
2-Being able to filter or prioritize feeds The problem is that I would love to suscribe to very diverse feeds, some would post maybe over a 100 post per day and others maybe one post every week or even month. So if let everything by default the former would flood the feed and I would never see the post from the little feeds. Here both offer categories that I could use but ideally I would love to have a curated main page. FreshRSS supposedly have a priority system but it seems quite simple and not effective for my needs, AFAIK you can put some feeds in "important feeds" but it only would show those feeds in that category then. TTRSS does have an advance filter system that is complex enough and with some fiddling I think I could make a set of rules that satisfy my needs. One point for TTRSS.
3-Being able to suscribe to any feed or even scrape webs that doesn't provide feeds. Here FreshRSS wins, I have zero issues subscribing to everything I wanted. With TTRSS I couldn't even subscribe with some pages that did provide with a feed, even if it was in an unconventional way. TTRSS devs say that is the webpage problem (even if FreshRSS had no problem with it). Here another point to FreshRSS.
And that is it, I do not exige that much. But I wasn't able to find a system that ticks those three checkboxes. FreshRSS was so close. But unless I am missing something you can't really create a curate feed that prioritizes and sorts feeds and posts in the way you can do with TTRSS sorting, if there is a way please let me know. And without that the whole thing becomes useless from the flooding feeds. And while I'm in love with TTRSS filters and sorting system, the whole app seems to unstable and with so many bugs to be usable, at least in my desired usercase (and I've seem many people complaining about TTRSS updates breaking things all the time).
My two main questions are:
-Am I missing some other self-hosted app that could do all I wanted?
-Am I missing some FreshRSS feature or extension that could curate a main feed with my own rules?
Any thoughts?
Do you have a proper robots.txt file?
Do they do weird things like invalid url, invalid post tries? Weird user agents?
Millions of times by the same ip sound much more like vulnerability proving than crawler.
If that's the case fail to ban or crowdsec. Should be easy to set up a rule to ban an inhumane number of hits per second on certain resources.