standard Online Security – Doing Better than Just Scraping by

1Data Thief – Hackers Cyber Crime” (CC BY-SA 2.0) by  perspec_photo88 

Web security is one of those issues that is relevant no matter how much hardware you are responsible for. Whether it is simply your personal laptop or the infrastructure of a whole organization, you can know for sure that there is someone somewhere who will be interested in getting inside your system. What happens beyond that point is a matter of scale, but in all cases it is not good news. No-one can afford to be lax when it comes to their online security.

WAF, the key line of defence

Ensuring that you have access to the latest Web Application Firewall (WAF) technology has never been more important. We are all familiar with the headline-grabbing cases of industrial-scale malpractice – in the UK the case of Carphone Warehouse in August 2015 was a high-profile example. But the gritty reality of life online is that smaller organizations are increasingly under threat as the hackers turn their attentions to what were once perceived as less well-defended targets.

There is an ongoing battle between governments, WAF developers and the criminally minded who are intent on malicious activity. The good news for smaller and medium-size businesses is that WAF is getting more affordable even as it becomes more sophisticated and more secure. For instance, it is effective at counteracting an increasingly common problem – the process of web scraping, which involves a bot infiltrating a website in order to extract either content or key data from the HTMl code or the underlying data base.

Mod_security and Mod_evasive for Apache” (CC BY 2.0) by  xmodulo 

An unhappy compromise

Since these bots frequently appear to introduce themselves to systems in the same ad hoc way as occasional human visitors, traditional security packages have struggled to contain them. The reason for this, in turn, is that these bots engage with the site in the same way as search engine algorithms. Stopping the malicious bots would mean ceasing all search engine operability – hardly a viable solution!

Web scraping itself is potentially damaging as it may lead to the loss of key data – as Carphone Warehouse discovered that can be extremely costly in terms of publicity and public reputation as well as cold hard cash – but it can also compromise security in subtler ways. For example, bots can run through a company database in order to discover confidential pricing information (known as ‘price scraping’). Equally damaging for some is the process known as ‘content scraping’, which involves entire databases being copied. Online business directories, for example, are particularly vulnerable to this sort of threat. In all cases the threat of scraping is one that may have a serious effect on the commercial viability of an organization.

Cutting-edge responses

To counter such measures security professionals have developed specific web-scraping security packages that operate by tracking the HTML fingerprints of each and every visitor to a site, by keeping records of the IP addresses of all previous threat sources, by analysing the behaviour of visitors to a site, and by presenting a series of progressive challenges that filter out human from non-human visitors. We’ve all been asked to recognise an alpha-numeric image for instance.

The ongoing battle between legitimate businesses simply seeking to use the web as a commercial medium, and those intent on exploiting any security deficiencies for unscrupulous ends continues. The good news is that whether you look to a cloud-based or a hardware solution, secure WAF security is doing more than just scraping by.

About the Author

Ali is a dental school graduate currently living in Amman, Jordan; born and raised in the US, a tech lover with a serious addiction to all things sweet. He is the co-founder of GeekOnGadgets, and a senior editor at our Sister site

Author Archive Page