Video Link Scraper

A three day project to help me keep up to date with the shows that I use to watch while living in Japan. By using Google's Custom Search API, I was able to consistenly parse data from multiple websites that I registered as 'custom search engine' through Google's CSE tool.

Initially, I planned on using a scraping library (Cheerio) to get the data that I needed. However, I quickly found out that certain sites will block GET requests if the agent is not a browser. Additionally, it would be too time intensive to create a scraper for each website that I wanted to use as a source. Google's Custom Search came to the rescue to overcome sites that blocked GET requests and it allowed me to focus on building one parser for multiple sources.

Role

• Full Stack Developer

Technology

• Node
• Express
• Angular
• Google API
• PostgreSQL
• HTML/CSS