Monday, October 13, 2014

Splunk Backup Script - splunkBackup.py

A while back I was working on a Splunk setup and needed to automate the backup and restore process. Since Splunk does not do this natively I thought I should whip-up something quickly. Currently going through the latest Splunk Admin guide (6.1.4) and noticed that there was still no option for automating the backup process. As a result I figured I should share this script as someone may either use it as it is or perfect it. Whichever works for you is fine with me.

The Backup Process
The script backups "/opt/splunk/etc" to the "/home/admin/s_backups" folder. Obviously you can change this. Maybe one of these days I will rewrite the script to allow you to choose your path. However, when this was done was to address a specific need.
It creates a log file of every backup which is done. The events are written to "/home/admin/s_backups/backup.log"

Sample backup log:
Sun Oct 12 21:25:48 2014: info:Backup /home/admin/s_backups/Sun_Oct_12_21_25_42_2014_BACKUP.tar.gz Completed successfully :root pid:19379 


Sample backup file "Sun_Oct_12_21_25_42_2014_BACKUP.tar.gz"

The Restore Process
The restore process restores the previous backup files by extracting the tar.gz file
It then replaces the existing "/opt/splunk/etc" folder
It then write a log entry similar to the backup entry

Email
You will need to specify your mail relay server
You will need to specify a receiving email 
Once the backup/restore process is completed, an email is sent to the address previously specified

Hope this script helps someone else



#!/usr/bin/env python
# This script makes a backup of the splunk 'etc' folder located in /opt/splunk/etc
# which contains all the Splunk configuration. It does not backup any data
# Author Nik Alleyne, CISSP | GCIA | SFCA 
# Email < nikalleyne at gmail dot com >
# splunkBackup.py
# March 2014

# To run this script manually, simple put an argument after the script. Any argument would do
# eg ./splunBackup.py --backup
# another example  even ./splunkBackup.py 0 will work also


import os
import shutil
import subprocess
import sys
import time
import tarfile

import smtplib
import email.utils
from email.mime.text import MIMEText

# Check the OS to ensure if is Linux
def check_os():
    if ( os.name == 'posix' ) and ( os.uname()[0] == 'Linux' ):
        print('=' * 50)
        #print(' Supported OS ')
        print(' :: OS: %s :: Host: %s \n :: User: %s :: PID:%s \n :: Time: %s ' %(os.uname()[0],os.uname()[1],os.getlogin(),os.getpid(),time.ctime()))
        print('=' * 50)   
    else:
        print(' While this script may work on other platforrms \n it was designed for Linux.')
        sys.exit(0)



# Build the menu
def menu():
    print('=' * 50)
    print(' Welcome to the Splunk Backup Script ')
    print(' Powered by SecurityNik ')
    print('=' * 50)
    print(' 1. Backup The System Config')
    print(' 2. Restore System Config ')
    print(' 3. Press any other key to exit  ')
    choice = raw_input(' \n Enter Selection:  ')

    #Read the choice and process
    if choice == '1':
        #print(' Beginning the backup process ... ')
        _backup()
    elif choice == '2':
        #print(' \n Beginning the restore process ...')
        _restore()
    else:
        print(' \n Exiting ... ')
        sys.exit(0)
   


   
# Do the work for backing up the configs
def _backup():
    BACKUP_DIR = '/home/admin/s_backups'
    BACKUP_LOG = '/home/admin/s_backups/backup.log'
    BACKUP_SRC = '/opt/splunk/etc'
   
    #print(' Beginning the Backup Process ')
    print(' \n Checking for backup directory ... %s ' %BACKUP_DIR)
    time.sleep(2)

    if ( os.path.exists(BACKUP_DIR) ) and ( os.path.isdir(BACKUP_DIR) ):
        print(' Backup directory found ')
        time.sleep(2)
    else:
        print(' Backup directory not found ')
        print(' Creating backup directory ...  %s '%BACKUP_DIR)
        try:
            os.mkdir(BACKUP_DIR,0755)
            f = open(BACKUP_LOG,'w')
            f.close()
            print(' Backup directory successfully created ')
            time.sleep(2)
        except:
            print(' An error occurred while creating the directory ... ')
            print(' Maybe a file currectly exists with this name in the /home/admin directory')
            print(' Try creating the directory %s manually and rerun the script ' %(BACKUP_DIR))
            sys.exit(0)
   
    # Write Backup information to log file
    f = open(BACKUP_LOG,'a')
    f.write('%s: info:Backup Started by user:%s pid:%s \n'%(time.ctime(),os.getlogin(),os.getpid()))
    f.close()
    time.sleep(2)

    print('\n Gathering files for backup .... ')
    print(' The following files and directories will be backuped up ')
    time.sleep(2)
    for root, dirs, files in os.walk(BACKUP_SRC):
        for d in dirs:
            print('%s' %root)
            for f in files:
                print('%s' %(os.path.join(root,f)))


    # Let's tar and zip the files in the /opt/splunk/etc folder
    try:
        BACKUP = tarfile.open(BACKUP_DIR + '/'+ '_'.join('_'.join(time.ctime().split()).split(':')) + '_BACKUP.tar.gz','w:gz')
        BACKUP.add(BACKUP_SRC, arcname='etc')
        BACKUP.close()
        print(' \n\n Backup completed successfully \n Backup stored in %s ' %(BACKUP.name))

        f = open(BACKUP_LOG,'a')
        f.write('%s: info:Backup %s Completed successfully :%s pid:%s \n'%(time.ctime(),BACKUP.name,os.getlogin(),os.getpid()))
        f.close()
    except:
        print(' An error occurred during the backup process. ')
        f = open(BACKUP_LOG,'a')
        f.write('%s: info:Backup %s ERROR!!! Backup not completed successfully :%s pid:%s \n'%(time.ctime(),BACKUP.name,os.getlogin(),os.getpid()))
        f.close()

   

def _restore():
    i = 0
    BACKUPS = {}
    BACKUP_DIR = '/home/admin/s_backups'
    BACKUP_LOG = '/home/admin/s_backups/backup.log'
    RESTORE_DIR = '/opt/splunk/'
    print(' Beginning the Restore Process ')
    print(' Locating backup directory ')
    time.sleep(2)


    # Write Restore information to log file
    f = open(BACKUP_LOG,'a')
    f.write('%s: info:Restore Started by user:%s pid:%s \n'%(time.ctime(),os.getlogin(),os.getpid()))
    f.close()
    time.sleep(2)


    if ( os.path.exists(BACKUP_DIR) and os.path.isdir(BACKUP_DIR)):
        print(' Backup dir found %s \n' %BACKUP_DIR )
    else:
        print(' Could not locate backup dir %s' %BACKUP_DIR)
        print(' You can also manually use tar to extract the file ')
        print(' Exiting ')
        sys.exit(0)
       
    for tar_file in os.listdir(BACKUP_DIR):
        if tar_file.endswith('_BACKUP.tar.gz'):
            i = i + 1
            BACKUPS[i] = tar_file
       
#    print(BACKUPS)
    for bak_no, bak_file in BACKUPS.items():
        print(' %d : %s' %(bak_no,bak_file))
   
    restore_choice = raw_input(' Please select a backup number:')
    if BACKUPS.has_key(int(restore_choice)):
        print(' Preparing to restore  %s ' %(BACKUPS.get(int(restore_choice))))
        RESTORE_FILE =  BACKUPS.get(int(restore_choice))
        #print(RESTORE_FILE)

    else:
        print(' Not a valid backup \n Exiting ... ')
        sys.exit(0)


    try:
        subprocess.call(['/opt/splunk/bin/splunk', 'stop'])
        os.chdir(BACKUP_DIR)
        if os.path.exists(RESTORE_DIR +'/etc.OLD'):
            shutil.rmtree(RESTORE_DIR +'/etc.OLD')
           
        os.rename(RESTORE_DIR +'/etc', RESTORE_DIR + '/etc.OLD')
        RESTORE = tarfile.open(RESTORE_FILE, 'r:gz')
        RESTORE.extractall(RESTORE_DIR)

        f = open(BACKUP_LOG,'a')
        f.write('%s: info:Restore %s Completed successfully :%s pid:%s \n'%(time.ctime(),RESTORE_FILE,os.getlogin(),os.getpid()))
        f.close()
        print('\n\n Restore completed successfully ')
        subprocess.call(['/opt/splunk/bin/splunk', 'start'])
    except:
        f = open(BACKUP_LOG,'a')
        f.write('%s: info:Restore %s Failed :%s pid:%s \n'%(time.ctime(),RESTORE_FILE,os.getlogin(),os.getpid()))
        f.close()
        print('\n\n Restore Failed to complete for %s ' %RESTORE_FILE)
        sys.exit(0)



def _mailer():
    BACKUP_LOG = '/home/admin/s_backups/backup.log'   
    SEND_FROM = 'SecurityNik Splunk <splunk@securitynik.com>'
    SEND_TO = 'Splunk Receiver <splunkreceiver@securitynik.com>'

    print(' Sending Mail ...')
    f = open(BACKUP_LOG)
    for line in f.readlines():
        pass
    print(line)

    msg = MIMEText(' Backup/Restore notification \n' + line + '\n Powered by SecurityNik')
    msg['From'] = SEND_FROM
    msg['To'] = SEND_TO
    msg['Subject'] = 'Splunk Backup/Restore Notification'

    _mailer_send = smtplib.SMTP('localhost')
    _mailer_send.sendmail(SEND_FROM, [SEND_TO], msg.as_string())

    f.close()
   



def main():
    subprocess.call('clear')
    check_os()

    if len(sys.argv) == 1:
        print(' Running in Automated Mode ... This mode works with cron. A cron job needs to be setup for this to be used')
        _backup()
    else:
        menu()

    _mailer()


if __name__ == '__main__':
    main()


splunkBackup.py
MD5: 6598a5c8f6de293b2a032972847e7288 splunkbackup.py
SHA1: 6e6a515ead39151873cb8b23a3d43c836f565cdc splunkbackup.py

4 comments:

  1. So much code, love this article

    ReplyDelete
  2. nice. I usually go with a bash script but interesting to see python version

    ReplyDelete
    Replies
    1. Thanks Anonymous! glad to see you are using batch script. The end justify the means :-)

      Delete