RasPI-Surv: a Raspberry-based scalable SMS remote controlled video surveillance system

Good morning dear security systems fanatics Garretlabs fans!

Today you will be very interested in my new post, I’m sure.

Because today we talk about a high-scalable videosurveillance system called RasPI-Surv, based on:

  • A system controller based on Raspberry PI
  • A practical (and modifications-friendly) command/telemetry interface via SMS messages, using the very well known SIM-900 module from ITEAD.
  • A set of wireless video cameras with good motion detection feature
  • A internet router/modem with wireless access point
  • A Google Mail account (it will be your “house account”), to be accessed via IMAP.

I engineered this system after a very strange request from one of my strangest friends. ūüôā

He was looking for a “internal” videosurveillance system with motion detection to be used only during his vacation periods…and after the vacation periods he wanted to disable the system and put all cameras and controller inside a box! ūüėČ

In other words he wanted a “portable” internal video surveillance system, without in-the-wall connections, without fixed positions for the cameras, so a very dynamic system with the opportunity to move the cameras and the controller inside his house. Plus, he wanted a powerful and portable alarm siren to be placed in his house, without external components (please note that this DIY solution is not accepted by italian laws, so in our country the alarm device must be always placed by a devoted an credited technician).

A practical example of the siren 12V powered could be THIS¬† …but you can buy another model following your needs.

Thinking for a smart solution, I studied a system with two day-night wireless IP Cameras from HooToo with pan and tilt features (http://www.hootoo.com/hootoo-ht-ip211hdp-indoor-wireless-network-camera-with-ir-cut-black.html) and two day-night D-Link fixed cameras (http://www.dlink.com/it/it/home-solutions/view/network-cameras/dcs-932l-day-night-cloud-camera). But you could add/remove wireless IP cameras with motion detection featire (also of different brands), in order to add/remove “eyes” to your system.

So, this is a very scalable solution. ūüėČ

Firstly I placed the cameras near a wall power socket (ok, they are wireless..but unfortunately they need power… ūüė¶ ), then I connected all cameras to the internet modem/router following their own user guides.

Finally I enabled their motion detection feature with the email sending (using the “house Google Mail account”) of the captured frame when a motion is detected.

Note that for some camera, a fast tramsition from day to night vision and viceversa (i.e. a sun ray refled by a passing car in the street) could be recognized as motion (so as a false alarm by the system ūüė¶ )…so the best way to place the camera is in front of closed external doors without glasses and of closed windows with the rolling shutter completely closed.

Well… do you placed the alarm siren and cameras? Have you configured your cameras in order to connect to your modem/router and to send an email using your Google mail account when a motion is detected? Very well… we can go ahead with the system idea.

This is the overview schema of the system (as always drawn by hand by me, sorry for the bad photo! ūüėČ ):

wpid-2014-11-10-08.37.09.jpg.jpeg

…In practice at this moment we have all cameras connected to our modem/router and a alarm siren connected to the power thru a relay commanded by the Raspberry PI. So, we must to setup the Raspberry PI in order to execute the following operations/requirements:

  • Periodically it should check the SENT folder of your Google Mail account, in order to verify if one of your IP cameras detected one motion (and so they sent an email using your “home account”). The cycle should be fast because one burglar can be very fast to open a window and to move in your house… ūüė¶
  • If one or more email arrived from your cameras, it should activate the relay in order to power ON the alarm siren
  • In this case it shoud send an SMS to your mobile phone in order to advise you that the siren has been powered ON
  • It should manage a set of SMS commands in order to: force the power OFF of the alarm siren, enable and disable the check of the sent emails, cancel the SMS memory of the SIM900 module, reset or reboot the system etc.
  • It should send a set of SMS in order to advise you that there is some problem (i.e. the internet connection is down, so the Google Mail mailbox can’t be accessed).

Well. It is’nt so difficult to implement it all. If you you read first these previous posts from Garretlabs: THIS ONE (it uses a GSM dongle and the software gammu in order to receive/send the SMS messages, plus to command the GPIO pins of Raspberry using Python) and also THIS ONE (it uses the SIM900 module and the AT commands in order to receive/send the SMS messages).

This is the Fritzing schema of the controller Raspberry PI+SIM900 module used to command the relay for the alarm siren:

Video surveillance controller (finale per uso con SIM900) v2_bb

I propose you the complete code of my project…. so you can now start playing with the RasPI-Surv system

 

#! /usr/bin/env python

import RPi.GPIO as GPIO
import imaplib, re, time, serial, subprocess

#class from http://segfault.in/2010/07/playing-with-python-and-gmail-part-1/ and http://segfault.in/2010/08/playing-with-python-and-gmail-part-2/
class pygmail(object):
    def __init__(self):
        self.IMAP_SERVER='imap.gmail.com'
        self.IMAP_PORT=993
        self.M = None
        self.response = None
        self.mailboxes = []

    def login(self, username, password):
        self.M = imaplib.IMAP4_SSL(self.IMAP_SERVER, self.IMAP_PORT)
        rc, self.response = self.M.login(username, password)
        return rc

    def get_mailboxes(self):
        rc, self.response = self.M.list()
        for item in self.response:
            self.mailboxes.append(item.split()[-1])
        return rc

    def get_mail_count(self, folder='Inbox'):
        rc, self.response = self.M.select(folder)
        return self.response[0]

    def get_unread_count(self, folder='Inbox'):
        rc, self.response = self.M.status(folder, "(UNSEEN)")
        unreadCount = re.search("UNSEEN (\d+)", self.response[0]).group(1)
        return unreadCount

    def get_imap_quota(self):
        quotaStr = self.M.getquotaroot("Inbox")[1][1][0]
        r = re.compile('\d+').findall(quotaStr)
        if r == []:
            r.append(0)
            r.append(0)
        return float(r[1])/1024, float(r[0])/1024

    def get_mails_from(self, uid, folder='Inbox'):
        status, count = self.M.select(folder, readonly=1)
        status, response = self.M.search(None, 'FROM', uid)
        email_ids = [e_id for e_id in response[0].split()]
        return email_ids

    def get_mail_from_id(self, id):
        status, response = self.M.fetch(id, '(body[header.fields (subject)])')
        return response

    def rename_mailbox(self, oldmailbox, newmailbox):
        rc, self.response = self.M.rename(oldmailbox, newmailbox)
        return rc

    def create_mailbox(self, mailbox):
        rc, self.response = self.M.create(mailbox)
        return rc

    def delete_mailbox(self, mailbox):
        rc, self.response = self.M.delete(mailbox)
        return rc

    def logout(self):
        self.M.logout()

#end library/class

##############################################################################
#Functions to PowerON OFF the relay
##############################################################################
#poweron the relay
def PowerONRelay():
    GPIO.setup(22,GPIO.OUT);
    GPIO.output(22,GPIO.HIGH);
    return;

#poweroff the relay
def PowerOFFRelay():
    GPIO.setup(22,GPIO.OUT);
    GPIO.output(22,GPIO.LOW);
    return;

##############################################################################
#Functions to manage SMS
##############################################################################
#check unread arrived messages
def CheckNewUnreadMessage(ser):
    print "Check for new messages..\n";
    ser.write("at\r");
    time.sleep(3);
    line=ser.read(size=64);
    #print line;
    ser.write('AT+CMGL="REC UNREAD"\r')
    time.sleep(3);
    response=ser.read(size=200);
    print response;
    return response;

#function to send confirmation SMS
def SendSMS(ser,value):
    #send SMS about the action
    ser.write("at\r");
    time.sleep(3);
    line=ser.read(size=64);
    print line;
    ser.write('AT+CMGS="+391234567890"\r'); #here put your phone number :-)
    time.sleep(3);
    if (value==1):
        ser.write('ALARM ON!\r');
    elif (value==0):
        ser.write('ALARM OFF!\r');
    elif (value==3):
        ser.write('ALARM enabled!\r');
    elif (value==4):
        ser.write('ALARM disabled!\r');
    elif (value==11):
        ser.write('ALARM commanded OFF (was ON)!\r');
    elif (value==20):
      ser.write('Deleted SIM memory!\r');
        elif (value==50):
      ser.write('INTERNET DOWN!\r');
    elif (value==51):
      ser.write('INTERNET UP!\r');
    elif (value==60):
      ser.write('Warning: GMAIL temporary DOWN!\r');
    time.sleep(3);
    ser.write(chr(26));
    return;

#function to delete messages...(GSM900 stores only 30 messages..)
def DeleteAllMsg(ser):
    ser.write("at\r");
    time.sleep(3);
    line=ser.read(size=64);
    print line;
    print "Deleting memory....";
    ser.write('AT+CMGD=1,4\r');
    time.sleep(3);
    return;    

###############################################################################
#main program for alarm check
###############################################################################
#global variables
initials_sent_emails=0
ALARM_enabled=True
ALARM_ON=False
actual_sent_emails=0;
ALARM_ON_cycles=0;
INTERNET_UP=True;
GMAIL_UP=True;
################################################################################
#functions to enable/disable/stop alarm (sent VIA SMS)
################################################################################
def enable_ALARM():
         global ALARM_enabled
         ALARM_enabled=True
         

def disable_ALARM():
         global ALARM_enabled
         ALARM_enabled=False
         

def stop_ALARM():
        global initials_sent_emails
        global actual_sent_emails
        #ALARM_ON=False
        initials_sent_emails=actual_sent_emails
        #ALARM_ON_cycles=0
        #TODO here send SMS telemetry

################################################################################
#MAIN program
################################################################################

#0: power ON the SIM900 module and waits for SIM registration
GPIO.setmode(GPIO.BCM)
GPIO.setup(17,GPIO.OUT)
GPIO.output(17,GPIO.HIGH)
print "Registering the sim...\n";
time.sleep(15); #wait for sim to register to the net
#0a: open serial link on /dev/ttyAMA0. It is open only one time:it remains open for all operation time.
#No more open+close action is now requested.
print "Opening communications serial...\n";
ser=serial.Serial('/dev/ttyAMA0',9600,timeout=1);
ser.open();


#1: first login to Gmail
g = pygmail()
g.login('your.home.mailbox@gmail.com', 'yourpassword')
print g.response

#2: list mailboxes
g.get_mailboxes()
for item in g.mailboxes:
  print item

#3: get number of sent emails at power ON of alarm controller...
g.get_mail_count('[Gmail]/Posta inviata')
print g.response
sent_number=g.response[0]
initials_sent_emails=int(sent_number) #actual number of sent messages
print initials_sent_emails

#4:logout from gmail
g.logout();


#Here we add the infinite loop (once per 10 seconds?) which:
#a. checks the number of sent emails (login+get_mails_count+logout)
#b. if the number is > initials_sent_emails sets (&& the alarm is ARMED) the alarm relay ON, else sets the alarm relay OFF
#b2. if this is the first time the alarm is ON, send SMS to communicate the ALARM is correctly ON!
#c. checks if a NEW SMS is arrived in order to enable/disable ALARM relay, and update the global variable
#d: waits for 10 seconds (TBC)
while (True):

       #a0: check for internet status
    ip="www.google.it";
        ret=subprocess.call("ping -c 1 %s" %ip,
        shell=True,
        stdout=open('/dev/null','w'),
        stderr=subprocess.STDOUT)
    if (ret==0):
         print "INTERNET UP!"
        if(INTERNET_UP==False):
            INTERNET_UP=True;
            SendSMS(ser, 51);
    else:
        print "INTERNET DOWN!"
        if (INTERNET_UP==True):
            INTERNET_UP=False;
            SendSMS(ser, 50);

    
       #a: check new sent messages
    if (INTERNET_UP==True):
            print 'Check new sent messages...'
        try:
                g.login('your.home.mailbox@gmail.com', 'yourpassword')
                g.get_mail_count('[Gmail]/Posta inviata')
                actual_sent_emails=int(g.response[0])
                g.logout();
            GMAIL_UP=True
        except imaplib.IMAP4.error, err:
            print 'GMAIL IMAP temporary unavailable!!!'
            if (GMAIL_UP==True):#the first time it sends an sms...
                SendSMS(ser,60)
                GMAIL_UP=False        

        #b/b2: alarm management
        print 'Alarm management...'
        if(ALARM_enabled==1 and actual_sent_emails>initials_sent_emails):
                print 'ALARM ON!!!'
                ALARM_ON=True
                ALARM_ON_cycles=ALARM_ON_cycles+1;
                #here we command the relay ON
                PowerONRelay();
                if (ALARM_ON_cycles==1):
                        SendSMS(ser,1);
                
        else:
                if(ALARM_ON==True):
                        SendSMS(ser,0); #if the alarm is already OFF, dont send messages...
                ALARM_ON=False
                ALARM_ON_cycles=0;
                #here we command the relay OFF
                PowerOFFRelay();
                print 'ALARM OFF!'

        #c: new SMS arrived management
        print 'SMS management'
    message=CheckNewUnreadMessage(ser);
    if(message.find("enablealarm")<>-1):
        enable_ALARM();
        #send message to my phone that Relay is ON
            SendSMS(ser, 3);
        print "Alarm enabled commanded\n";    
    elif (message.find("disablealarm")<>-1):
        disable_ALARM();
        SendSMS(ser, 4);
        print "Alarm disabled commanded\n";
    elif (message.find("alarmoff")<>-1):
        stop_ALARM();
        #send message to my phone that telemetries are OFF
          SendSMS(ser, 11);
        print "Deactivated alarm\n";
        elif (message.find("delsms")<>-1):
        DeleteAllMsg(ser);
        SendSMS(ser,20);

       
        #d: basic cycle time
        print 'Wait for new cycle...'
        time.sleep(2);

 

Weeeeeeell….I Hope the code is well commented,so it is not so unreadable! ūüėČ

As you see, I used a very nice Python library in order to connect to Google Mail using IMAP. You can download it from Segfault.in, and you can find good examples in the following pages: http://segfault.in/2010/07/playing-with-python-and-gmail-part-1/  and http://segfault.in/2010/08/playing-with-python-and-gmail-part-2/.

…Anyway, if you want to request some detail…I am here for you! ūüôā

And as usual, if you want to comment/modify/correct/redistribute/relink my code and my idea, you ‘re free to do it (…and thanks for the your pingbacks and for credits in your blogs etc. ūüėČ )!

Bye bye geek guys and girls… it’s time for me to go to install and test the system in my friend’s house (also if it’s too late for his vacation time..ah ah!!!)! ūüôā

 

Advertisements

Raspberry and the remote controlled relay: a low-level approach (a.k.a. “AT modem commands: the usual suspects”).

Hi my dear microelectronics F&M (female & male) friends! ūüėČ
Here we are again with the remote controlled 220V relay using a Raspberry PI. In my previous post I used a Huawei 3G dongle with the famous gammu daemon as SMS receiver.
But… do you remember the end of the post? I was talking about a thriller/horror movie by the great Lucio Fulci.
So, the horror turned to real, since gammu-smsd, after 20-25 minutes of correct activity, decided to die himself, without any clear cause. Or better, it continued running in background but when a SMS arrived, the daemon did nothing.
I tried to “reanimate” it in several ways, removing the screensaver and the blank screen from the Raspberry, writing a script which stop/restart the daemon every 5 minutes etc. But nothing changed. ūüė¶
I noticed also after the above mentioned 20-25 minutes, the daemon start/restart (using sudo /etc/init.d/gammu-smsd start/restart) fails.
Only a reboot works in this case….but NO PANIC please! ūüôā

I think there is some problem in gammu-smsd (related to Raspbian memory management? Hmmmm…I dont want investigate it because I want to live again ūüėČ ).
So….I decided to try another approach, using a more simple hardware and writing myself a more simple (= with a simple maintance and debug phases) software.
I bought the ITEAD GSM Sim900 module for Raspberry PI from my usual store (Robot Italy). Price: 37 euros. I think it is a good price for a GSM/GPRS phone without keys and display! ūüėČ
I placed my SIM in the module, then I placed the module on the Raspberry.
The module reports the unused GPIOs on the higher side, so I connected the three wires used to command the relay (Vcc, GND and command) directly to the SIM900 module.
Note that the Vcc available on the SIM900 module is 3.3V , but I noticed that my relay module works well also using 3.3V instead the requested 5V. ūüôā
Note also that GPIO18 used to command the relay in my previous post, is used now by SIM900 module as software reset, so I modified my relay ON/OFF scripts in order to use GPIO22.

This is the schema of SIM900 module used GPIOs:Mappatura RasPi GPIO-ITEAD SIM900
Ok, let’s go powering the Raspberry board (with a 2A minimum power supplier, since the SIM900 can use also peaks of current of 1A).
But firstly I needed the right soundtrack for this activity, and the right soundtrack in this case is “The City Sleeps”, a powerful and aggressive crossover-progressive rock cd, in my opinion the best work of Touchstone, the band of my friend Kim Seviour.

Let’s return to us… now the Raspberry boots up but the SIM900 module is powered off (no green led is ON on the surface).
There are two ways to power it on:

  • the first approach is to simply press the power on button :-). Too simple for me.;-)
  • the second way is to set the GPIO17 high….well, you know this is my preferred solution. ūüėČ

The python code (remember you must before install python extensions devoted to GPIO commanding, following my previous post!) is the following:

#poweron the RPI GSM SIM900 module
def PowerON_SIM900_module():
    GPIO.setup(17,GPIO.OUT)
    GPIO.output(17,GPIO.HIGH)
    return;

Raspberry talks with SIM900 module via a serial link mapped on /dev/ttyAMA0, but Raspberry locks this port at startup. The document reporting the correct actions to free the serial port (before to try to talk with the module) are in ITEAD provided C library…but they are written in chinese! ūüė¶
So I googled aroud a lot before to find the solution to this problem, damn! ūüė¶
This is the correct procedure in order to “unlock” /dev/ttyAMA0 port (application note taken from ITEAD blog, related to another ITEAD product):

In /etc/bootline.txt change:

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p6 rootfstype=ext4 elevator=deadline rootwait

toÔľö

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p6 rootfstype=ext4 elevator=deadline rootwait

and in /etc/inittab comment out or delete the following line (in my case it is the last line):

# T0: 23: respawn :/ sbin / getty-L ttyAMA0 115200 vt100

Well…after this, a reboot is necessary.

The Itead GSM module...with "The City Sleeps" of Touchstone!

The Itead GSM module…with “The City Sleeps” of Touchstone!

Now, I tried using the examples provided in the official C library from ITEAD, but I encountered some error (very strange errors, such as “SIM not registered, some unpredictable error code and so on…), so I decided to talk directly to SIM900 module using AT commands.

Firstly I installed the old, well known minicom software then I connected minicom to SIM900 module with the following (the “old style values”) parameters: port: /dev/ttyAMA0, speed: 9600 bps, no flow control, no parity, 8 bit per word, 1 stop bit.

I sent some famous AT commands, such as “AT” (in order to verify if the momdem is responding), “AT+CMGS” (in order to send to my phone a text SMS), etc. and I verified that all worked well.

Quasi-incredible! ūüôā

At this point I decided to implement (in Python, of course) the main code, which: receives the SMS message (many thanks to this tutorial for inspiration of this part), parses them and, if a valid command is received, it powers ON/OFF the relay. Plus, the script can send to my phone  a SMS reporting the action (Power ON/OFF) executed.

I used only three AT simple commands:

  • AT (in order to verify if the communication between module and Raspberry is OK)
  • AT+CMGS (used to send SMS messages)
  • AT+CMGL= “REC UNREAD” (used to read received unread messages)

Note that I added some delay, since I noticed that in some case the modem is a little slow to execute the AT commmands and to reply. This is the complete Python code.

But first don’t forget to install the python extension for serial communication using sudo apt-get install python-serial.

#! /usr/bin/env python

import serial
import time
import RPi.GPIO as GPIO

GPIO.setmode(GPIO.BCM)

#poweron the RPI GSM SIM900 module
def PowerON_SIM900_module():
    GPIO.setup(17,GPIO.OUT)
    GPIO.output(17,GPIO.HIGH)
    return;

def PowerONRelay():
    GPIO.setup(22,GPIO.OUT);
    GPIO.output(22,GPIO.HIGH);
    #send message to my phone that Relay is ON
    SendSMS(1);
    return;

def PowerOFFRelay():
    GPIO.setup(22,GPIO.OUT);
    GPIO.output(22,GPIO.LOW);
    #send message to my phone that Relay is OFF
    SendSMS(0);
    return;


def SendSMS(value):
   #send SMS about the action
   ser=serial.Serial('/dev/ttyAMA0',9600,timeout=1);
   ser.open();
   ser.write("at\r");
   time.sleep(3);
   line=ser.read(size=64);
   print line;
   ser.write('AT+CMGS="+39123456"\r');
   time.sleep(3);
   if (value==1):
       ser.write('Relay powered ON!\r');
   elif (value==0):
       ser.write('Relay powered OFF!\r');
   time.sleep(3);
   ser.write(chr(26));
   ser.close();
   return;

def CheckNewUnreadMessage():
   print "Check for new messages..\n";
   ser=serial.Serial('/dev/ttyAMA0',9600,timeout=1);
   ser.open();
   ser.write("at\r");
   time.sleep(3);
   line=ser.read(size=64);
   #print line;
   ser.write('AT+CMGL="REC UNREAD"\r')
   time.sleep(3);
   response=ser.read(size=200);
   print response;
   ser.close();
   return response;

#main loop
   PowerON_SIM900_module();
   print "Registering the sim...\n";
   time.sleep(15); #wait for sim to resgister to the net
   print "Started main loop\n";
   while True:
      message=CheckNewUnreadMessage();
      if(message.find("poweron")<>-1):
          PowerONRelay();
          print "PowerON commanded\n";
      elif (message.find("poweroff")<>-1):
          PowerOFFRelay();
          print "PowerOFF commanded\n";
     time.sleep(5);

As you see from the code, the new, unread SMS messages are checked every 5 seconds.

If a new message is found, it is copied in the message variable. If the message contains the poweron keyword the relay is powered ON. Else,  if it contains the poweroff keyword, the relay is powered OFF.

In both cases a SMS is sent to my phone (well… this is not my real number, also if I know you would like to know it! ūüėČ )

In order to auto-start the script after the Raspberry boot, you should add this command to /etc/rc.local (prior, remember to “executabilize” ūüėĬ† your script using the old chmod +x name_of_your_script.py ):

sudo /home/pi/name_of_your_script.py &

IMPORTANT NOTE: Add the “&” in order to launch in background the Python script, or you won’t be able to stop it neither using CTRL+C or swithcing shell . I will re-take the control of your Raspberry only via remote connection (using ssh)! ūüôā

Now, rebooting the Raspberry, you will be able to receive/send SMS message using AT commands, power ON/OFF a 220V relay…and the most important thing: I tested the system for 3+ hours, and it worked well all time!!!

So ….I think we executed today another litlle step in my and in your “knowledge database system” called BRAIN. ūüôā

Now I will take a relax time with the aid of the new Touchstone cd: “Oceans of Time”. It’s an kick-ass prog-rock album, of highest level guaranteed (as we say in Italy, it’s “Lemon guaranteed”, that is “Garantito al limone”).

Bye bye Raspberry freaks, it’s rock time for me! ūüėČ