I found FediSearch, and also this post basically saying that a fediverse search engine would just be used as a tool by trolls.
this post was submitted on 21 Dec 2023
273 points (97.9% liked)
Fediverse
32478 readers
641 users here now
A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).
If you wanted to get help with moderating your own community then head over to [email protected]!
Rules
- Posts must be on topic.
- Be respectful of others.
- Cite the sources used for graphs and other statistics.
- Follow the general Lemmy.world rules.
Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)
founded 2 years ago
MODERATORS
It's worth noting that since FedSearch, Mastodon has actually natively implemented opt-in search on posts.
That’s a good point. But those people can be banned? I guess Reddit handles this by moderation and archiving old posts.
Yes, but moderation teams on the fediverse are very small, and by nature of it, can make hundreds of account of different servers all trailing that would need to be individually sought out and banned.
It is a game of cat & 100 mice
load more comments
(3 replies)
That post wasn't claiming that a search engine would only be used by trolls; it was explaining that they shut down their project because a chunk of the fediverse thinks that and complain about any search engine projects. Discoverability is one of the network's biggest challenges and a search engine could really help with that.
Yes, not only used by trolls, but would be a tool that could be leveraged by trolls. And I think the fediverse makes it easier to establish instances for marginalized groups, but also has more admins that just don't want trolls because nobody here is making $ off them like the corporate socials are. I think if adding search that is going to try and vacuum up everyone's posts in the fediverse and make them easily sortable/targetable without instance admins permission, then that isn't cool. If someone is running a general instance that covers nothing that a troll could latch onto and wants the instance catalogued and searchable then that's fine by me. I don't think boys should be doing that to the fediverse as a whole without admin permission though.
I don't think an admin's permission has anything to do with it. If you post publicly on the fediverse, your posts are public. You should have the option to opt out of any indexing (just like you do for the rest of the open web). But saying its ok for you to read this post if it happens to come across your feed but you shouldn't be allowed to find it via a search is ridiculous. Users get to make the choice with each post whether its public or not, but they don't get to control how people consume those public posts.
load more comments
(3 replies)
load more comments
(13 replies)
Well, please make sure it respects post privacy at least but also realize that on the microblogging side of the fediverse, they may not take kindly to this prospect at all. People who start these kinds of projects are often harassed or at least receive passive hostility. Making it opt in instead out of opt out in some capacity is best.
A good search engine would be quite important. One thing that annoyed me back on the site that should not be named was that their search engine was completely useless - It was not even capable to find posts where I entered verbatim text of.
Having a good search engine that can actually find a post I was looking for would be a major plus for the fediverse.
Yep, the idea is to simulate the type of results you get from Google. People trust Lemmy answers more than spam sites now a days.
Why wouldnt people want do have search engine? Without it Fediverse stands no chance against non-free internet. Everything posted here would be much more valuable if it was searchable. Now comment posted once is viewed only until post gets less popular. Any other site of this kind displays answers decades old. Privacy isnt issue as everything posted here is available to everyone on internet.
I mean they are posting on the public internet, they should know that it can be read by anyone. I like the idea of users opting out.
If you give users the choice to opt out, all the privacy-focussing communities won't be searchable.
What if someone who opts out posts a comment and someone who opts in answers?
The Fediverse is public, a search engine doesn't show anything that isn't already open for anyone to see.
Good point. They should know they are making public comments. If you want it private then send a private message.
Is this something you can point yacy at?
You should federate the search engine so that folks can defed from the search as desired.
But then we would need a search engine for all the search engines...
Everything you post on the Fediverse is public.
If you don't want to show up in an internet search, post your stuff on your private server and only give access to the people you want to invite.
Yep
There is already Sepiasearch specifically for Peertube.
It's limited to only Peertube and it's not the most intuitive. I want to work with them on expanding this.
I love the idea, especially from a technical standpoint!
How big is the fediverse today? How many posts are there? What kind of algorithms atmre you using to store the results? Do you scan sites and then their connected sites or do you have a premade list?
More technical information please 😊!
The fediverse is a few thousand servers, from Mastodon, Lemmy, etc. Can't say the amount of posts but there are a lot.
So on the more technical side, I plan on using a light weight fast search engine called Sonic (It's written in rust). I have already used it in other projects and it can handle billions of messages / posts. But it has a cost it doesn't have faceted search, like for example if you want to exclude certain texts from the results. I think this is a fair trade off. The other solution would be to use something more mature like ElasticSearch but it'll be expensive (I'm assuming not much money will be made from this and I'm talking about donations)
For scanning sites there are premade lists to start with and it'll be possible to scan new sites from other instances if found. So a bit of both.
I support bigger picture. Rather than an independent site, wouldn't it be more practical to work with current fediverse app developers for lemmy, mastodon, etc to integrate search engine within the app?
I’m reaching out to see their thoughts. But there are limitations to what they can index.
The fedi elite will snipe you from a bell tower.
view more: next ›