Twitter Scraper

Easily scrape Twitter(X) public data within minutes using Crawlbase. No risk of IP blockage.

Trusted by 70,000+ users

Twitter (X) Scraper

Examples of scraping use cases

Scrape Followers

Get to know the audience

Scrape Hashtags

Usage data for hashtags

Scrape Profiles

Collect usernames, bios, profile pictures, work, education and more

Scrape URLs

Get links shared on Twitter

Scrape image/videos

Retrieve image and video links, captions, and engagement metrics

Scrape Trends/Trending Topics

Explore public sentiments and popular discussions with trends

Live Twitter Scraper Demo

👇🏼 Try it now, paste a website URL

JSON result:


{
  "url": "https://twitter.com/elonmusk",
  "body": {
    "name": "Elon Musk",
    "username": "@elonmusk",
    "joinedDate": "June 2009",
    "location": "",
    "coverPhoto": "https://pbs.twimg.com/profile_banners/44196397/1690621312/600x200",
    "profilePhoto": "https://pbs.twimg.com/profile_images/1780044485541699584/p78MCn3B_200x200.jpg",
    "description": "",
    "about": {},
    "followingCount": "582",
    "followingCountFull": 582,
    "followersCount": "181.1M",
    "followersCountFull": 181182550,
    "totalTweetsCount": 43144,
    "scrapedTweetsCount": 7,
    "tweets": [
      {
        "text": "Absolutely true! “The evidence is overwhelming that the majority of people on the street are there because of untreated mental illness or addiction, which leads people to use all their money to support their drug habit and be high, rather than work.”",
        "images": [
          "https://pbs.twimg.com/ext_tw_video_thumb/1783539713602908163/pu/img/ZwF7UGafjx--ICbt.jpg"
        ],
        "video": null,
        "replyCount": "6K",
        "retweetCount": "15K",
        "likeCount": "72K",
        "viewCount": "14M",
        "datetime": "2024-04-25T23:43:24.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1783643000759091344"
      },
      {
        "text": "the bird is freed",
        "images": [],
        "video": null,
        "replyCount": "135K",
        "retweetCount": "372K",
        "likeCount": "2.3M",
        "viewCount": "",
        "datetime": "2022-10-28T03:49:11.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1585841080431321088"
      },
      {
        "text": "Listen, I can’t do miracles ok",
        "images": [
          "https://pbs.twimg.com/media/FRZViwWX0AMsqQ1?format=jpg&name=small"
        ],
        "video": null,
        "replyCount": "73K",
        "retweetCount": "221K",
        "likeCount": "2.4M",
        "viewCount": "",
        "datetime": "2022-04-28T01:57:27.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1519495982723084290"
      },
      {
        "text": "Let’s make Twitter maximum fun!",
        "images": [],
        "video": null,
        "replyCount": "108K",
        "retweetCount": "212K",
        "likeCount": "2.4M",
        "viewCount": "",
        "datetime": "2022-04-28T01:53:50.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1519495072802390016"
      },
      {
        "text": "Next I’m buying Coca-Cola to put the cocaine back in",
        "images": [],
        "video": null,
        "replyCount": "183K",
        "retweetCount": "797K",
        "likeCount": "4.5M",
        "viewCount": "",
        "datetime": "2022-04-28T00:56:58.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1519480761749016577"
      },
      {
        "text": "Yesss!!!",
        "images": [
          "https://pbs.twimg.com/media/FRNsuSFWUAUW6aP?format=jpg&name=small"
        ],
        "video": null,
        "replyCount": "142K",
        "retweetCount": "381K",
        "likeCount": "2.4M",
        "viewCount": "",
        "datetime": "2022-04-25T19:43:22.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1518677066325053441"
      },
      {
        "text": "I hope that even my worst critics remain on Twitter, because that is what free speech means",
        "images": [],
        "video": null,
        "replyCount": "170K",
        "retweetCount": "410K",
        "likeCount": "3M",
        "viewCount": "",
        "datetime": "2022-04-25T16:12:30.000Z",
        "tweetLink": "https://twitter.com/elonmusk/status/1518623997054918657"
      }
    ]
  }
}

Try it Now

Top reasons for companies choosing Crawlbase Twitter Scraper

Great Support & Docs

Average success rate: 100%

GDPR & CCPA Compliant

Seamless Scaling Power

No Limits, Global Proxies

Easily Bypass Captchas

Crawlbase is the best solution to scrape millions of Twitter pages

Our Twitter scraping API is created to operate across a vast network of residential and data center proxies globally. Our infrastructure guarantees anonymity and complete security against IP blocks and overcoming CAPTCHAs that are a common challenge when scraping Twitter.

We provide easy integration to start with. The default rate limit is 20 requests per second but can be increased per you scraping needs. The constant IP rotation acts as a shield against IP bans and ensure uninterrupted access to unlimited data. We offer customized solutions to meet your unique requirements, providing a personalized approach to enhance your web scraping.

Try it yourself

Overview of Crawlbase's Twitter Scraper

User-friendly interface for easy usage, with limited coding knowledge

Highly scalable API with global proxies

Automate browser scraping for JavaScript-heavy websites

Protect Web Crawler from blocks, proxies, IP leaks, crashes, and CAPTCHAs

Export data in various formats like CSV, Excel, and JSON

Fetch accurate and high-quality data

Real-time data in a 4-10 seconds response time

Unlimited bandwidth with complete documentation

Start crawling in minutes

All-in-one solution for collecting Twitter data

Crawlbase keep you safe from IP blocks and CAPTCHAs. Use our Crawling API to get the full HTML code and scrape any content that you want. Take a screenshot of an entire Twitter page on a high resolution if you wish to keep track of any visual changes quickly with our Screenshots API. Send your crawled pages straight to the cloud using Crawlbase’s Cloud Storage. For huge projects, you can use the Crawler with asynchronous callbacks to save cost, retries, and bandwidth.

Create Free Account!Contact Sales

Frequently Asked Questions

Is it illegal to scrape Twitter data?

Twitter scraping is a legal gray area. Twitter's terms of service prohibit web scraping, but the legality can vary by location and the purpose of scraping.

How do I scrape Twitter data in Python?

To start with a Python Twitter scraper, a recommended approach is to utilize the Crawlbase Crawling API. Start by setting up an account on Crawlbase, getting your private token, and installing the Crawlbase Python library. Develop a Python script to interact with the Crawling API and retrieve HTML content from Twitter. For targeted information extraction, incorporate the "autoparse" parameter, which streamlines data retrieval by providing key details in a JSON format. Efficient storage, analysis, and visualization of data can be achieved using Python libraries like Pandas, Matplotlib, and Seaborn. This enables users to gain insights from Twitter posts, comments, and user interactions.

I want my requests to come from a different country, is that possible?

Yes, you can use the country parameter if you want your requests to be geolocated from a specific country. By default, your account will have access to more than 25 countries that you can use anytime on each of your API requests.

What is the rate limit of your API?

The default rate limit for most websites is 20 requests per second. However, if you need to scale it up to meet your production needs, you can easily contact us to discuss your rate limit increase.

In what format is the data delivered?

Crawlbase is designed to give users Twitter data in a structured format, mostly JSON, because it's simple and works well in web development. JSON organizes data with key-value pairs, making it easy to understand and analyze. To get detailed information about how the data is formatted, you can check Crawlbase's documentation or contact their support team.

What does the free trial include?

Crawlbase offers your first 1000 requests free of charge, allowing you to test their services without requiring payment information upfront. Simply sign up, explore the capabilities, and decide whether it suits your needs before committing to any payment.

Why is it important to have an unblocking solution when scraping?

Twitter web scraping comes with challenges like legal constraints, technical limitations (rate limiting and IP blocking), data accuracy, and ethical concerns. Therefore, unblocking measures are one of the first things to consider. However, platforms like Crawlbase offer effective solutions, ensuring compliance with Twitter’s policies and optimizing the scraping process to minimize risks and efficiently gather data within acceptable boundaries.

Start crawling the web today

Try it free. No credit card required. Instant set-up.

Start crawling in minutes