Loading ...
Sorry, an error occurred while loading the content.
 

Creating a new /etc/hosts file to block ads and malware

Expand Messages
  • thad_floryan
    Many of you are probably weary of my commenting how nice it is to surf the web without any ads and not to have to worry about malware from ads like those that
    Message 1 of 3 , Apr 30, 2012
      Many of you are probably weary of my commenting how nice it is to surf
      the web without any ads and not to have to worry about malware from ads
      like those that infected 100000s of readers of the web New York Times
      recently. Additionally, there is a recent increase in the number of
      so-called "hacked" Yahoo account sending spam and malware URLs to the
      various groups; that's caused by infected PCs (from malware ads using
      IFRAMEs and other ilk) keylogging and sending user names and password to
      the malware authors.

      We had a thread about that yesterday in another [astronomy] group and I
      showed them how to block ads (and infections) and I'm going to share the
      same information here.

      There are two sites I've been using in this regards for ages:

      <http://www.mvps.org/winhelp2002/hosts.htm>
      and
      <http://someonewhocares.org/hosts/>

      The trick is to uniquely combine the ad and malware IPs from both sites
      and incorporate that along with any "local" additions into /etc/hosts
      which can be done with a simple script as I'll show below.

      Here's an example of the components of the procedure to automatically
      create a new /etc/hosts file (which I ran in a temporary directory a
      few minutes ago):

      -rw-r--r-- 1 thad thad 707968 Apr 30 06:14 combined
      -rw-r--r-- 1 thad thad 709106 Apr 30 06:14 hosts
      -r--r--r-- 1 thad thad 1138 Apr 30 05:51 LAN_IPs
      -r-xr-xr-x 1 thad thad 1437 Apr 30 06:17 make_hosts
      -rw-r--r-- 1 thad thad 585545 Apr 30 06:14 MVPS
      -rw-r--r-- 1 thad thad 61 Apr 30 06:14 MVPS_updated
      -r--r--r-- 1 thad thad 273 Apr 30 06:07 OTHERS
      -rw-r--r-- 1 thad thad 302967 Apr 30 06:14 SWC
      -rw-r--r-- 1 thad thad 40 Apr 30 06:14 SWC_updated

      The LAN_IPs file is simply the IPs and hostnames of equipment on my
      LAN; for example:

      # LAN_IPs
      #
      127.0.0.1 localhost localhost.localdomain localhost4
      ::1 localhost localhost.localdomain localhost6
      [...]
      192.168.100.1 modem
      [...]
      172.20.20.1 router
      [...]
      172.20.20.50 regulus
      172.20.20.51 antares
      172.20.20.52 sirius
      172.20.20.53 arcturus
      [...]
      172.20.20.80 lanserv1
      172.20.20.81 lanserv2
      172.20.20.82 lanserv3
      [...]
      172.20.20.85 lancam1
      172.20.20.86 lancam2
      [...]
      172.20.20.90 tl2015
      172.20.20.95 usbhub
      172.20.20.98 wap
      172.20.20.99 tl4050

      The OTHERS file contains IPs of sites I choose to reject and the
      preamble to the main list of ad and malware IPS, for example:

      # auto-start video sites used by SFGate beginning Sept. 2011
      #
      127.0.0.1 truveo.com
      127.0.0.1 xml.truveo.com
      127.0.0.1 www.truveo.com

      # ad sites to ignore per combined unique URLs from:
      #
      # http://winhelp2002.mvps.org/hosts.htm, and
      # http://someonewhocares.org/hosts/
      #

      and here's my "simple" script to fetch the ad and malware list, keep
      track of their update dates, and create a new hosts file to be later
      moved to /etc/hosts; note the line beginning "cat MVPS SWC | ..." has
      been folded with a "\" due to YAHU's line-length bugs:

      #!/bin/bash
      # make_hosts
      # script to create a new hosts file for /etc from a local file
      # of LAN IPs, a local file of IPs to ignore, and an optimized
      # list of ad and malware IPs from two public-service web sites.
      #
      # Author: Thad Floryan
      #
      ########################################
      # - get MVPS hosts.txt data file
      # - convert to Linux/UNIX newlines
      # - track the update date
      #
      wget http://winhelp2002.mvps.org/hosts.txt
      dos2unix -n hosts.txt MVPS > /dev/null
      rm hosts.txt
      grep -i "updated:" MVPS >> MVPS_updated

      ########################################
      # - get SWC's hosts data noting this wget
      # fetches an index.html file
      # - track the update date
      #
      wget http://someonewhocares.org/hosts/
      mv index.html SWC
      grep -i "updated:" SWC >> SWC_updated

      ########################################
      # for all lines in both MVPS and SWC beginning with a "127"
      # - convert all spaces and tabs (:blank:) to a single space (\040)
      # - sort the lines
      # - eliminate duplicate lines using 'uniq'
      # - save results in the file named 'combined'

      cat MVPS SWC | grep "^127" | tr -s '[:blank:]' '[\040*]' | \
      sort | uniq > combined

      ########################################
      # create the new hosts file from the concatenation of:
      # - local LAN's hosts template (LAN_IPs)
      # - local list of any other sites to ignore (OTHERS)
      # - combined MVPS and SWC lists of ad and malware sites
      #
      cat LAN_IPs OTHERS combined > hosts
      echo "Your new hosts file is ready; happy surfing!"
    • roro
      why the sound of that email smells faster than a hallelujah on rails!
      Message 2 of 3 , May 1 11:27 AM
        why the sound of that email smells faster than a hallelujah on rails!
      • thad_floryan
        ... Hi Alanna, You didn t attribute and/or cite anything [above] so I take that as meaning you liked the script for creating a new hosts file. Am I correct?
        Message 3 of 3 , May 1 5:47 PM
          --- In linux@yahoogroups.com, roro <roro@...> wrote:
          >
          > why the sound of that email smells faster than a hallelujah on rails!

          Hi Alanna,

          You didn't attribute and/or cite anything [above] so I take that as
          meaning you liked the script for creating a new hosts file.

          Am I correct? :-)
        Your message has been successfully submitted and would be delivered to recipients shortly.