Ad

Download Files From SFTP Server That Are Older Than 5 Days Using Python

- 1 answer

I got a Python script on this site that downloads files from the directory from SFTP server. Now I need help to modify this code so that it only downloads the files that older than 5 days from the day the code is used.

Code to download files (based on Python pysftp get_r from Linux works fine on Linux but not on Windows):

import os
import pysftp
from stat import S_IMODE, S_ISDIR, S_ISREG

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None    
sftp=pysftp.Connection('192.168.X.X', username='username',password='password',cnopts=cnopts)

def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
    for entry in sftp.listdir(remotedir):
        remotepath = remotedir + "/" + entry
        localpath = os.path.join(localdir, entry)
        mode = sftp.stat(remotepath).st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath,mode=777)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath, preserve_mtime)
        elif S_ISREG(mode):
            sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

remote_path=input("enter the remote_path: ")
local_path=input("enter the local_path: ")

get_r_portable(sftp, remote_path, local_path, preserve_mtime=False)

Please help me to modify the code so that it only download files 5 days older from the present day.

Ad

Answer

Use the pysftp.Connection.listdir_attr to get file listing with attributes (including the file timestamp).

Then, iterate the list and pick only the files you want.

import time
from stat import S_ISDIR, S_ISREG
def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
    for entry in sftp.listdir_attr(remotedir):
        remotepath = remotedir + "/" + entry.filename
        localpath = os.path.join(localdir, entry.filename)
        mode = entry.st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath, preserve_mtime)
        elif S_ISREG(mode):
            if (time.time() - entry.st_mtime) // (24 * 3600) >= 5:
                sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

Though the code can be much simpler, if you do not need a recursive download:

for entry in sftp.listdir_attr(remotedir):
    mode = entry.st_mode
    if S_ISREG(mode) and ((time.time() - entry.st_mtime) // (24 * 3600) >= 5):
       remotepath = remotedir + "/" + entry.filename
       localpath = os.path.join(localdir, entry.filename)
       sftp.get(remotepath, localpath, preserve_mtime=True)

Though note that pysftp seems dead. Consider using Paramiko instead. It has pretty much same API, so the above code will work almost as it is (except that Paramiko does not have the preserve_mtime parameter). See also pysftp vs. Paramiko.


Based on:

Ad
source: stackoverflow.com
Ad