Weather station update scripts

Introduction

• These describe the pipeline used to create the Weather Station website and database
  
• Data from the weather stations is uploaded several times a day to HOBOlink servers

Automated data analysis (using cron)

Explanation

• To run shell script without cron -> ./microclimate_cron.txt
• To see and edit cron job in personal crontab -> crontab -e
• How it works - The shell script/cron job:
  
• Downloads most recent copy of update script from Google Doc (using lynx to strip markup)
• Runs the database update and figure update scripts in R batch mode
• (To do: Send email updates to notify of errors http://kangry.com/topics/viewcomment.php?index=427)
  

Shell script

Microclimate station metadata

• Purpose: Store, update, and serve data on weather station location, sensor types, and data format etc...
  

Database update script

• Purpose: Update MySQL server with most recent data available on HOBOlink servers
• Identifies new data, then organizes it and inserts into MySQL table
  
• Written in R and MySQL (using RMySQL interface)
  

db: Editable script

db: Extracted (run) script

db: R outfile

Figure/summary update script

• Purpose: Outputs figures and tabular summary of most recent microclimate data
• Written in R and MySQL (using RMySQL interface)
• The files output by this script are then directly read by the Weather Station plone page
  

fig: Editable script

fig: Extracted (run) script

fig: R outfile