Multiprocessing for heavy API requests with Python and the PokéAPI

While working on a recent project, I realized that heavy processes for python like scrapping could be made easier though python’s multiprocessing library. The documentation and community engaging in multiprocessing is fairly sparse, so I wanted to share some of my learnings through an example project of scrapping the PokéAPI. Below I wrote a bit of code that pulls all of the available pokémon while minding the API’s 100 calls per 60 second limits. You’ll see that the iteration is fairly slow as there are 964 pokémon the API returns.

Read the full story

Read more

%d bloggers like this: