While most of the people here upset and rightfully so, we have got to move past the angst to actually build this place out.
We have to recreate the environment and communities we’ve created on on Reddit here, so that people don’t feel like they’re missing out being on kbin.
That way, the next time Spez goes full Elon, we already have everything in place to make the jump.
I help run r/MMA on Reddit. We aggregate news as well as bringing on MMA figures for AMAs.
We, like many other magazines here, will need a way to quickly aggregate breaking news onto our magazine, and the easiest way to do that is going to be through a bot that mirrors submissions to our subreddit.
At least for the mags that rely on breaking news, if we implement this at scale, the end user wont be missing much by migrating over here if they get all the same great breaking news.
Do we have any bot builders in the house who could take this on as a project?
We would be leveraging Reddit’s own userbase at scale to better kbin, and eventually beat them in the long run. That’s the biggest L we could ever deal them.
EDIT: WASN’T SURE THIS WAS GONNA BLOW UP. IVE MADE https://kbin.social/m/BotIt FOR DEVELOPMENT. IF YOU’RE INTERESTED AND CAN HELP OR JUST WANT TO FOLLOW THE PROJECT, COME ON OVER. THANKS.
Devs (like me) can help, the main issue is hosting the bot somewhere and covering the costs, but I think is a good idea.
Maybe create a magazine for it to get some additional support? I could edit the top post.
I will take a look at the technical side, I want to know how easy it is to make something like this. I won’t make promises but if manage to get something working I’ll let you know.
I don’t think basic scraping will be particularly difficult, especially if the rates are kept low. While I also don’t think it’s actually the right way forward, I’ll happily help out.
Even if it was just the top upvoted link post per sub each day, that would give a lot of content here to get started and make communities (or magazines kbin parlance) look alive. Once people start posting on their own it won’t be needed anymore.
The API prices are high, but maybe it can be done with just one or two calls, which would make it almost nothing. Of course, the API might just disappear one day.
Fuck the API, scrape the HTML from old.reddit.com for free
https://kbin.social/m/BotIt if you would be willing to help
If Reddit has proven anything as of late, they’re unreliable as a partner.
I’m clearly not a developer but would have reservations in relying on the API just for how erratic they are.
Nah, I mean more that I think relying on people to just post stuff themselves would be better for the community. But scraping would be the way forward, relying on the API would be prohibitively expensive IMO.
You think using the api is better? I’ll take a look at it
No, no, I think it would ultimately be better to rely on people to post stuff. But scraping links would be better than using the API.
Pro Tip:
instead of linking to magazines with their links, do it something like this:
@BotIt@kbin.social@BotItThis way no matter what server you’re viewing from, you’ll stay logged in.
I made https://kbin.social/m/BotIt for us to congregate
What are the decentralized p2p storage options for hosting bots, would decentralized hosting and storage options be a viable solution?
You can get VPS’s for like $30 a year these days.
I keep hearing this but all the VPS solutions I find cost an arm + leg. What VPS solutions cost $30/year?
Vultr has VPSs starting at $2.50/mo ($30/yr)
I also commonly find links to really cheap servers on nerdvittles.com
Example:
https://my.racknerd.com/cart.php?a=add&pid=735
$13/mo
The best thing we can do is be active here. Make the effort to get involved, post , report idiots and be civil. This goes beyond reddit, it is about getting out from under the thumb of corporate control. We can’t have free expression if the narritive is being controlled.
100% agree
our social media sites need to be open source and decentralized. any other way will lead to the slow inevitable march of enshittification
Also we have networks here we can draw upon, devs , artists , folks who make videos, people who write, etc we can. Ask them for help to create videos and promotional material to quickly explain the fediverse in a simple and concise way.
Mirroring would be trivial, but you’d have to either scrape content yourself or pay for the API (soon)
I’ve built scrapping bots like this before. Would definitely be an interesting project.
https://kbin.social/m/BotIt lets do it!
It’s trivially easy, but you’d be pulling in a lot of noise along with the signal, creating more moderation headaches for yourself (think of all the low-effort and spam stuff you usually have to filter out). You’d be better off scraping the content you want from primary sources directly rather than mirroring every post that goes to your old forum.
RSS would do the job
https://kbin.social/m/BotIt made this to figure it out
You raise a good point. Little overhead, the endpoints are well-formatted, you can get a digest of articles in one blow without API keys, and you just need to parse the resulting XML.
I think the key is you don’t actually mirror every single post, just posts that meet specific guidelines like so many upvoted and no self posts or reddit images
That seems reasonable, although there is no telling what a highly voted post might constitute under new management (is that too paranoid?). I’d personally take a scrap and build approach here, or at least manually approve the incoming results (hybrid approach) if they’re being delayed anyway due to waiting on vote generation.
I’ve thought about it, but I’m not sure if it’s legally allowed or if reddit dmcas you. Would be down to join the effort, it shouldn’t be impossible. But compute heavy.
Just don’t index the reddit self posts or images hosted on reddit. They can’t tell us not to index links to other people’s websites.
There’s no way to enforce that. Bots can change IP address and not all instances have a dmca page.
Do not make this place a mirror of reddit.
I’m not even talking about the legality of mirroring another site. You really want to give ammo to the CEO of reddit and allow him to claim that his competitors are attempting to mirror his content?? Can you imagine how he is going to use this if you do it?
Just post your own content, if you have a real interest into MMA then it should be easy.
We have to recreate the environment and communities we’ve created on on Reddit here, so that people don’t feel like they’re missing out being on kbin.
The “fear of missing out” is something you should not listen to. Do not replicate the content of reddit, why would you do that? Just create your own content. If it’s news then read your sources and post the news.
We, like many other magazines here, will need a way to quickly aggregate breaking news onto our magazine
Well, post something if you find it interesting. And let other people post what they think is interesting. You don’t need no machinery for that. We want quality content, not the quantity delivered by a spambot machine, which in the ends are adbots, posting advertised stuff like the endless NBA stuff. If you read nothing interesting in the news then post nothing in your subs.
We must not duplicate what reddit did wrong. And the 3rd party tool is not the only thing that reddit did wrong. What reddit did wrong is content for the sake of content, for the thumb constantly swiping down. This is not good for reddit and this is not good for kbin. This is good for no one.
The quality and content of reddit collapsed, do not duplicate it here.
We want the creators, not the followers. Do not listen to the followers who are addicted to new content. I don’t want people just watching the endless ribbon of news, posting a “lol” or a one liner from time to time. Are you sure that you are not the one who fears of missing out? Are you dependent on popularity or somethin’? Because it surely looks like so.
What next? You want to mirror publicfreakout so we again watch american people fight into airplanes and governors banning books on the other side of the planet? Because this is what you’re gonna get.
edit: And we didn’t even talk about the COST of handling automated, mirrored content. If reddit cannot pay for it then neither can Ernest.
edit: If all of you who upvoted OP instead posted something they find interesting in the subs they follow then we would not have this discussion.
You want to mirror publicfreakout so we again watch american people fight into airplanes and governors banning books on the other side of the planet? Because this is what you’re gonna get.
what is so wrong with this? the reason reddit is successful is because there are infinite genres of content. i think you make a strong argument that we shouldn’t mirror reddit…
but people posting videos like /r/fightporn or /r/crazyfuckingvideos or whatever I think is perfectly fine and should be encouraged
what is so wrong with this?
What’s wrong? There are people fighting around the world all the time. This is not content, this is distraction. And now there is distraction specially created FOR publicfreakout, with people picking fight just so they can have a video and post it to reddit. This is just an example of what is wrong. Do you want this “content” here? I don’t.
If we mirror reddit: first we won’t be able to pay the bill, this argument alone should discredit the mirror thing. Second, we will encounter the same deadweight problem that redit encountered.
Please post content that matters to you, not to what you think the audience want.
I like the idea of making a bot that posts news links automatically. Gives fresh content for people to interact with, without people racing over each other to get the karma of posting them first. Communities based on sports events are the simplest to do that with, you just need to set the bot to take news from some specific big sites and bam you got stuff to talk about.
In practice it’s not so easy without some manual curation. News sites post a lot of filler stuff and you don’t want to start spamming yourself with every article posted to <insert magazine here>. Even on higher-traffic subs you don’t generally see more than one or two posts from the same site on a given day. It’s generally more effective with something repeatable and reliable like a weekly column where the expected “quality” is invariate. Certainly you can front-load the manual curation by building a set of filters into your scraper, but whether you filter the results at the front or the end of the pipe, you still need some kind of heuristic for what constitutes “good” content, and that’s frequently a moving target.
I don’t know about the rest of you, but even working a full time mid-career corporate job, I think I can make time for both. It doesn’t have to be either-or.
Won’t that become a problem when API access is highly restricted? Generally it is doable and should not be a very big project.
I’m clearly not an expert on this, but I know people scrape data without API’s
Of course it’s possible, but API would be the quickest and most simple solution.
Nah, bot scraping is different than API calls
Yes, you are right. My question pointed more in the direction whether an API solution is doable since it might be easier.
Technically I think? I think the free API license gets 100 calls a minute? Or an hour or something, I forget the time frame. On a medium sub, 100 posts a day would be updates every 15 minutes 24/7 I think. Not instant, but absolutely doable, and if you optimize the time for 6am-1am, that gives you one about every 12 min?
Great idea! I really miss r/MMA
I think it’s mainly about introducing new users to the fediverse and making it easier to understand, I’m glad I found out about this stuff and am happy to learn more.
I think it is for a lot of magazines, however our magazine and subreddit requires timely news to really get going.
If you want an even dirtier approach, you’d make a browser extension that scrapes reddit as your browse it and forwards links you upvote to kbin / lemmy or whatever. This has the benefit of being indistinguishable from regular browser traffic as far as reddit is concerned so they won’t notice.
Ah it seems I wasn’t the only one who thought about this - https://kbin.social/m/RedditMigration/t/38958/Bot-that-will-repost-reddit-content-to-kbin-fediverse#entry-comment-159480 !!! Fully support this idea, will join the magazine and see how I can contribute to mirroring content from Reddit to Kbin.
Yep it was my 2nd ever post on here from a week ago: https://kbin.social/m/kbinMeta/t/10949/Could-Someone-Build-a-Bot-that-Mirrors-Subreddit-Submissions#comments
Glad this one took off and we can hopefully build this. Fingers crossed.
draft - This is a good post. Should/can I post this post link with a short description to my mastodon account? What about my twitter account?
