I discovered that thousands of bots monitor every new link posted on X
I discovered that 1000s of bots are actively monitoring X for new links posted, probably not just on my account
I noticed when everytime I'd share one of my sites new features or pages, the page would get a bit slower just after posting, then I asked Claude to analyze it
I tweeted a test URL as a reply somewhere deeply hidden in a thread (so not even a main post)
And within 8 seconds there's thousands of crawlers opening the page!
One is a military intelligence firm, that scans social media for breaking news and sells alerts to newsrooms, banks, companies and government, it specifically follows accounts with many followers
Another one is a social media monitoring company that tracks what people say online
I knew of course high frequency trading firms use online news to trade fast on things, but interesting to see it in real life
Twitterbot (X) of course is first, within 0 seconds of sharing, it opens it (probably when I wrote the tweet)
The first non-X bot arrives just 8.40s later, which I think is pretty fast considering it's a reply by me deeply hidden in a thread somewhere
TL;DR they're watching you!
If it's a main tweet (so not a reply), it's even faster
Within 0.4 seconds there is non-X crawlers opening the link!
Originally posted on X · 💬 Reply to this on X
P.S. I'm on 𝕏 too if you'd like to follow more of my stories. And I wrote a book called MAKE about building startups without funding. See a list of my stories or contact me.
Never miss a post: 183,778 people read my posts in full by email, before it spreads anywhere else
