Thursday, July 28, 2011

Simple Script to Download epaper from The Hindu


#!/bin/bash
#ishan dot karve at gmail dot com
#
#Script to download epaper from Hindu
#No more subscription .. pls donate the money to Prime Ministers Welfare Fund
#As always /// Its free to use...
#Get user to select edition
edition_choice=([0]=101 [1]=102 [2]=103 )
edition_name=([0]=Chennai [1]=Hyderabad [2]=Delhi )
echo "Hindu epaper editions are"
echo "-------------------------------------------------"
echo "0. Chennai"
echo "1. Hyderabad"
echo "2. Delhi"
echo "-------------------------------------------------"
while true; do
    read -p "Enter edition you wish to selec[0-2]: " ed
    case $ed in
       [012]) 
 echo "Thanks." 
 break;;
        * ) echo "Please select the correct numeric serial.";;
    esac
done
#spider the selected edition using wget to estimate number of pages
#define max incremental page limit
max_spider=100
echo "Estimating number of pages in ${edition_name[ed]} edition"
#start spider for main editon
 for ((  j = 1 ;  j <= $max_spider;  j++  ))
    do
      #prepend zero to single digits
      pageno=`printf "%03d" $j`  
      echo "Searching for Page $pageno"
      I_FILE="http://epaper.thehindu.com/pdf/`date +%Y`/`date +%m`/`date +%d`/`date +%Y``date +%m``date +%d`A_$pageno${edition_choice[ed]}.pdf"
 debug=`wget --spider $I_FILE 2>&1`
 
 echo $debug
      if [[ $debug =~ .*link!!!.* ]]
      then
      break
      fi
done
clear
#decrement counter
(( j = j - 1 ))
npages_A=$j
echo "Estimating number of pages in ${edition_name[ed]} edition supplement"



#start spider for newapaper supplement
 for ((  j = 1 ;  j <= $max_spider;  j++  ))
    do
      #prepend zero to single digits
      pageno=`printf "%03d" $j`  
      echo "Searching for Page $pageno"
      I_FILE="http://epaper.thehindu.com/pdf/`date +%Y`/`date +%m`/`date +%d`/`date +%Y``date +%m``date +%d`B_$pageno${edition_choice[ed]}.pdf"
 debug=`wget --spider $I_FILE 2>&1`
 
 echo $debug
      if [[ $debug =~ .*link!!!.* ]]
      then
      break
      fi
done
clear
#decrement counter
(( j = j - 1 ))
npages_B=$j

ty_dir="$HOME/Desktop/hindu_${edition_name[ed]}_`date +%d``date +%m``date +%Y`"
#mkdir to store individual pages
mkdir $ty_dir
echo "Please be patient..Bandwidth intensive operation starts..;-)"
echo "Downloading Main Paper .. total $npages_A pages"
    for ((  i = 1 ;  i <= npages_A;  i++  ))
    do
      #prepend zero to single digits
      pageno=`printf "%03d" $i`  
      echo "Downloading Page $pageno"
      O_FILE="$ty_dir/A$pageno.pdf"
      I_FILE="http://epaper.thehindu.com/pdf/`date +%Y`/`date +%m`/`date +%d`/`date +%Y``date +%m``date +%d`A_$pageno${edition_choice[ed]}.pdf"
      wget -q -O $O_FILE $I_FILE 
      
    done
    
echo "Downloading Supplement .. total $npages_B pages"

  for ((  i = 1 ;  i <= npages_B;  i++  ))
    do
      #prepend zero to single digits
      pageno=`printf "%03d" $i`  
      echo "Downloading Page $pageno"
      O_FILE="$ty_dir/B$pageno.pdf"
      I_FILE="http://epaper.thehindu.com/pdf/`date +%Y`/`date +%m`/`date +%d`/`date +%Y``date +%m``date +%d`B_$pageno${edition_choice[ed]}.pdf"
      wget -q -O $O_FILE $I_FILE 
      
    done
   

echo "Combining all pages into a single pdf document"
#combine multiple pdf files
gs -dNOPAUSE -sDEVICE=pdfwrite -sOUTPUTFILE=The_Hindu_${edition_name[ed]}_`date +%d``date +%b``date +%Y`.pdf -dBATCH $ty_dir/*.pdf
#empty directory
rm $ty_dir/*.*
#remove directory
rmdir $ty_dir
How to get it running

Copy the script to your Linux desktop
 go to command prompt using terminal
 type following commands

cd ~/Desktop
chmod +x thehindu.sh
./thehindu.sh

Tuesday, July 19, 2011

Simple Script to Download epaper from Mid-Day


#!/bin/bash
#ishan dot karve at gmail dot com
#Script to download epaper from mid-day.com
#As always /// Its free to use...
#Get user to select edition
edition_choice=([0]=mumbai [1]=delhi [2]=bangalore [3]=pune)
edition_abbr=([0]=md-mn [1]=md-dn [2]=md-bn [3]=md-pn)
echo "Mid-Day epaper editions are"
echo "-------------------------------------------------"
echo "0. Mumbai"
echo "1. Delhi"
echo "2. Bangalore"
echo "3. Pune"
echo "-------------------------------------------------"
while true; do
    read -p "Enter edition you wish to selec[0-9]: " ed
    case $ed in
       [0123]) 
 echo "Thanks." 
 break;;
        * ) echo "Please select the correct numeric serial.";;
    esac
done
#Get user to input starting page
read -p "Please enter the starting page you wish to download from?" strt_pg
#Get user to input ending page
read -p "Please enter the ending page you wish to download?" end_pg
while true; do
    read -p "Do you wish download pages $strt_pg to $end_pg? [Y/N]" yn
    case $yn in
        [Yy]* ) 
    for ((  i = $strt_pg ;  i <= end_pg;  i++  ))
    do
      echo "Downloading Page $i"

      I_FILE="http://epaper2.mid-day.com/DRIVE/${edition_choice[ed]}/`date +%d``date +%m``date +%Y`/epaperpdf/19072011-${edition_abbr[ed]}-$i.pdf"
      wget $I_FILE 
    done
     break;;
        [Nn]* ) exit;;
        * ) echo "Please answer yes or no.";;
    esac
done







How to get it running

Copy the script to your Linux desktop
 go to command prompt using terminal
 type following commands

cd ~/Desktop
chmod +x milk_day.sh
./milk_day.sh

Simple Script to Download epaper from Indian Express

#!/bin/bash
# ishan dot karve at gmail dot com
#Script to download epaper from indian express
#As always /// Its free to use...
clear
curl -s http://epaper.indianexpress.com > /tmp/editions
temp1=$(sed -nr 's/(.*)max-height:none;" src="?([^ ">]*).*/\2\n\1/; T; P; D;' /tmp/editions) 
temp2=$(sed -n -e 's/.*<span class="caption">\(.*\)<\/span>.*/\1/p' /tmp/editions)
editions=($(echo $temp2 | tr " " "\n"))
editions_link=($(echo $temp1 | tr " " "\n"))
echo "Following ${#editions[*]} Editions available for download"
count=0
for i in "${editions[@]}"
do
 echo $count.  $i
 count=$((count+1))
done
while true; do
    read -p "Enter edition you wish to select[0-9]: " ed
    case $ed in
       [0123456789]) 
echo "Processing..." 
break;;
        * ) echo "Please select the correct numeric serial.";;
    esac
done

if [ "$ed" -ge "${#editions[*]}" ]
then
echo "Please select proper edition. Please try again. Bye."
exit 0
fi

links=${editions_link[$ed]}
#grab edition id
edition_id=($(echo $links | cut -d "/" -f4))
#Get user to input starting page
read -p "Please enter the starting page you wish to download from?" strt_pg
#Get user to input ending page
read -p "Please enter the ending page you wish to download?" end_pg
while true; do
    read -p "Do you wish download pages $strt_pg to $end_pg? [Y/N]" yn
    case $yn in
        [Yy]* ) 
 ty_dir="$HOME/Desktop/ie_day_`date +%d``date +%m``date +%Y`"
 mkdir $ty_dir

 for ((  i = $strt_pg ;  i <= end_pg;  i++  ))
 do
#prepend zero to single digits
      pageno=`printf "%02d" $i`  
      echo "Downloading Page $pageno"
      O_FILE="$ty_dir/$pageno.pdf"
      I_FILE="http://epaper.indianexpress.com/pdf/get/$edition_id/$i"
   wget -O $O_FILE $I_FILE 
 done
  break;;
        [Nn]* ) exit;;
        * ) echo "Please answer yes or no.";;
    esac
done
#combine multiple pdf files
gs -dNOPAUSE -sDEVICE=pdfwrite -sOUTPUTFILE=ie_`date +%d``date +%m``date +%Y`.pdf -dBATCH $ty_dir/*.pdf
#empty directory
rm $ty_dir/*.*
#remove directory
rmdir $ty_dir



How to get it running

Copy the script to your Linux desktop
 go to command prompt using terminal
 type following commands

cd ~/Desktop
chmod +x milk_express.sh
./milk_express.sh

Tuesday, July 12, 2011

Simple Script to Download epaper from Times of India

#!/bin/bash
#Written for a friend in need
#Script to download epaper from indiatimes.com
#As always /// Its free to use...
#Get user to select edition
edition_choice=([0]=TOIM [1]=CAP [2]=TOIB [3]=TOIKM [4]=TOICH [5]=TOIPU [6]=TOIA [7]=TOIL [8]=TOIJ [9]=TOIH)
echo "Times of India epaper editions are"
echo "-------------------------------------------------"
echo "0. Mumbai"
echo "1. Delhi"
echo "2. Bangalore"
echo "3. Kolkata"
echo "4. Chennai"
echo "5. Pune"
echo "6. Ahmedabad"
echo "7. Lucknow"
echo "8. Jaipur"
echo "9. Hyderabad"
echo "-------------------------------------------------"
while true; do
    read -p "Enter edition you wish to selec[0-9]: " ed
    case $ed in
       [0123456789]) 
 echo "Thanks." 
 break;;
        * ) echo "Please select the correct numeric serial.";;
    esac
done
#Get user to input starting page
read -p "Please enter the starting page you wish to download from?" strt_pg
#Get user to input ending page
read -p "Please enter the ending page you wish to download?" end_pg
while true; do
    read -p "Do you wish download pages $strt_pg to $end_pg? [Y/N]" yn
    case $yn in
        [Yy]* ) 
    for ((  i = $strt_pg ;  i <= end_pg;  i++  ))
    do
      echo "Downloading Page $i";
      I_FILE="http://epaper.timesofindia.com/Repository/${edition_choice[ed]}/`date +%Y`/`date +%m`/`date +%d`/${edition_choice[ed]}_`date +%Y`_`date +%-m`_`date +%d`_$i.pdf"
      wget $I_FILE 
    done
     break;;
        [Nn]* ) exit;;
        * ) echo "Please answer yes or no.";;
    esac
done


How to get it running

Copy the script to your Linux desktop
 go to command prompt using terminal
 type following commands

cd ~/Desktop
chmod +x test_1.sh
./test_1.sh


14 Jul 2011 0026 : Script updated to reflect various editions....

Saturday, July 9, 2011

Microcontroller based Pump Controller

Updates:-
  1. 28 Jun 11: I cant help but thank the kind guys at Texas Instruments. I cant believe my eyes, they have shipped me a sample of their instrumentation amplifier. Thats would help me simplify my input block. Thanks u guys… You rock.
  2. 01 Jul 11: Received Sample. Waiting for Farnell guys to ship my pressure sensors…
  3. 07 Jul 11 :  Received Pressure sensors.
 
 
This project will describe building of a Pump control system. The requirement primarily arose to satiate the requirements of my parents, who have to manually switch on the pump every day morning and switch it off and continuously monitor the tank for overflow. So I have taken up the challenge onto myself...
However I would be ungrateful, I do not acknowledge the inputs I received after reading Kayne Richens blog post.
 
This blog is progressive ; hence will get updated as things progress.....

Lets me see how do things unfold...


Basic Design Objectives:-
  1. Should be cheap and affordable.
  2. Should be as automated as possible requiring minimal user supervision.
  3. Should have a user-friendly MMI.
  4. Prevent water and electricity wastage.
Design Challenges:-
  1. Erratic water supply timings.
  2. Non scheduled Electrical Load shedding .
  3. High head of water tank.
  4. Intelligent control system.
Preliminary System Design:-

image


System Blocks Functions:-
Input Block:-
Input Description  Sensor Type  Micro-controller Input  
Sense Tank Capacity
Differential Pressure Sensor
Analog
Sense Electricity Availability Optocoupler Digital/Interrupt
Sense Time RTC Digital
Sense Input Water Supply Availability Differential Pressure Sensor Analog

Output Block:-
Output Description  Transducer Type  Micro-controller Output  
Motor On/Off
Solid State Relay
Digital
Valve Solenoid On/Off Relay Digital
Alarm Buzzer Analog

 
 
Mathematics for Selecting Differential Pressure Sensor:-
 
Why Differential Pressure Sensor?
 
A differential pressure sensor will eliminate variation in pressure due to changes in atmospheric pressure.
 
Mathematics
 
Capacity of Water Tank = 500 Lt
Water Tank Head = 30 Ft = 9.144 Mt = 914.4 Cm
Average Ambient Temperature = 25 deg Celcius
Specific Gravity of Tap water=0.997gm per cubic cm = 997Kg per cubic meter
Dia of Discharge Pipe=1/2 in = 1.27 Cm=0.0127 M

Pressure exerted at Base of Building by a column of Water at height
 
P = h * rho * g
where
P = Pressure
h = height of liquid column (2000 mm = 2m)
rho = Density of water (1000 kg/m^3)
g = Acceleration due to gravity (9.81 m/s^2)
 
P=9.14 x 997x 9.81 = 89394.4098 Pascals = 89.39kPa = 89.4 kPa (approx)
Similarly Pressure at Rooftop at the base of tank itself
P=1.5 x 997x 9.81 =14670.855 Pascals =14.67kPa = 14.7 kPa (approx)
 
Selected Sensor  :  Freescale Differential Pressure Sensor MPX2100DP  0-100 kPa


Selection  of MCU:-

Family : Microchip PIC
Why : Because I have necessary experience and required development tools.

Max No of ADC Channels Required = 3 (2xpressure Sensor + 1 spare)
No of Interrupts : 02 (Electrical Supply Available & Float Valve)
Digital I/O = 8
Operating Voltage= 5V
SPI Required = Yes (For Interfacing RTC)
MCU Selected  = 16F876A  or 16F877A Why? Because I already have!

Design of Pressure Sensor Amplifier:-
Design Guide/Reference : -
  1. Op Amps for Everyone Design Guide.

Amplifier Design Objective : The output span of the transducer must be matched to the input span of the ADC to achieve optimum performance.

Mathematics for Amplifier Design

Span and Offset Characteristics at 10v,15v & 5V Excitation Voltage. As sensor is ratio-metric the the values are scaled for.

MPX2100DP

Excitation Voltage Min Span 
(mV)
Max Span 
(mV)  
Min. Offset 
(mV)  
Max Offset 
(mV)  
10V
38.5
40
-1
1
15V (scaled)
57.75 60 -1.5 1.5
5V (scaled)
19.25
20 -0.5 0.5
9V(scaled) 34.65 3.6 -0.9 0.9
 
 
MPX2010DP
 

Excitation Voltage Min Span 
(mV)
Max Span 
(mV)  
Min. Offset 
(mV)  
Max Offset 
(mV)  
10V
24
26
-1
1
15V (scaled)
36 39 -1.5 1.5
5V (scaled)
12
13 -0.5 0.5
9V(scaled) 21.6 23.4 -0.9 0.9
 
 

Desired Amplified Span and Offset


Desired Offset= 0.5V
Desired Span = 5V

Gain Range & Offset Calculation
Maximum Gain      =Desired Span (V)
                           Sensor’s Minimum Span
Desired Span = 5V

Excitation Voltage

Maximum Gain (MPX2100DP)

Maximum Gain (MPX2010DP)

10V =5/38.5 = 129.8 = 130

208

15V =5/57.75 = 86.58 = 87

139

5V =5/19.25 = 259.75 = 260

417

9V =5/34.65=144.300=144

231

 
Since Texas Instruments was kind to ship me samples of their instrumentation amplifiers, I would be basing my calculations on same. There is no rocket science involved since the datasheet is fairly self explanatory and the instrumentation amplifier itself requires only one external component for gain adjustment.
Texas Instruments was generous in mailing following samples to me
INA126, INA128,INA122,INA2126
 
I would be using INA126 amplifier for testing purpose
 
Brief description about INA126 ..”The INA126 and INA2126 are precision instrumentation amplifiers for accurate, low noise differential signal acquisition. Theirtwo-op-amp design provides excellent performance with very low quiescent current (175µA/channel). This, combined with awide operating voltage range of ±1.35V to ±18V, makes them ideal for portable instrumentation and data acquisition systems. Gain can be set from 5V/V to 10000V/V with a single external resistor. Laser trimmed input circuitry provides low offset voltage (250µV max), low offset voltage drift (3µV/°C max) and excellent common-mode rejection.”
 
Gain calculation formulae
 

Instr Amplifier

Gain Formula

INA 122 G=5+200kΩ 
         RG
INA 126 G=5+80kΩ 
         RG
INA 128 G=1+50kΩ 
         RG
 
RG (IN OHMS)values at Max Gain. Values in (brackets) indicate nearest standard resistor values.
 
MPX2100DP
 

Instr. Amplifier

10V

15V

5V

9V

INA 122

   1602 Ω (1.6KΩ)

2452Ω(2.4KΩ)

785Ω (750Ω)

1439Ω(1.5K)
INA 126

641Ω (620Ω)

981Ω (1kΩ)

584Ω (560Ω)

576Ω(560Ω)
INA 128

388Ω (390Ω)

548Ω (560Ω)

193Ω (180Ω)

347Ω(360Ω)
 
MPX2010DP
 

Instr. Amplifier

10V

15V

5V

9V

INA 122

   984 Ω (1KΩ)

1494Ω(1.5KΩ)

486Ω (470Ω)

885Ω(820Ω)

INA 126

393Ω (390Ω)

598Ω (560kΩ)

194Ω (200Ω)

356Ω(360Ω)

INA 128

241Ω (240Ω)

363Ω (360Ω)

120Ω (120Ω)

214Ω(200Ω)

 
Amplifier Interface Design
 
\includegraphics[scale=0.500000]{ckt8.1.1.ps}
 
In the MPX sensor, each of the resistors in the bridge is an active strain gage, so this design illustration is applicable for the amplifier design.
 
image
 
Test Schematic
 
image
 
3D PCB Render (Using Diptrace)
 
image