Screen Scraping Security

By John Wagley

The issue is growing in importance as tools to scrape screens for data are becoming more common and powerful, said Joanne Furtsch, policy and product architect at TRUSTe. Whereas much market data and research used to be collected by telephone, such data collection has been surpassed by online-based research, according to Furtsch.

Many organizations’ marketing, public relations, or research departments may be either considering getting involved in or already engaged in this type of data collection, she said. Privacy officers and other executives at those companies should make sure that those running the program know what kind of data can be legally collected. It’s also important to monitor what may be collected by any third-party research firms. Businesses must avoid infringing on other companies’ privacy policies and terms of use, she said. If any policies are unclear, the organization needs to get clarification before it proceeds with the data collection at that site.

Executives should also assess whether data being collected may be sensitive or personally identifiable information under state and national national laws, said Bowman. Companies should consider applying filters that can remove names from data.

Social networking sites and blogs should be sure to let consumers know, in their privacy policies and other areas, how the information they share on the site could be collected, said Furtsch. Such sites should also let potential screen scrapers know what information they’ll allow to be scraped. Some sites, such as Facebook, forbid any kind of automated data collecting, even if it’s by a user collecting data from his or her own account.


When deciding how to

When deciding how to implement your social media marketing strategies, it is important to take into consideration the nature of your products and services. For example, if purchasing your products is something that most of your customers would prefer to keep private, then do not put Facebook-like buttons right next to the buy buttons! Eventually, someone will click it accidentally and then get angry at your business.

Great Article but needs more security solutions


Great article! The one thing I believe missing is more effective ways to protect a website from screen scraping. Robots.txt unfortnantly doesn't work. Luckily, there are several companies out there that offer or specialize in Scraping protection.  Services such as,, or can put a barrier to entry to prevent scrapers from taking your content. 


Great article ! Many bots

Great article !

Many bots are very sophisticated, changing their IP and its hard to track them

SiteBlackBox  can help you to stop such abuse and keeps those bots away for good


The Magazine — Past Issues


Beyond Print

SM Online

See all the latest links and resources that supplement the current issue of Security Management magazine.