# function set assumes AWS credentials are already set locally import boto3 import botocore session = boto3.Session() S3 = session.resource('s3') # case insensitive find objects function # all kw must match for a target match # any ignore kw is an ignore match def findObjects(bucket, TARGET_KW, IGNORE_KW = []): my_bucket = S3.Bucket(bucket) filelst = [] for obj in my_bucket.objects.all(): ignoreflag = 0 matchcount = 0 # adjust case for proper matching filename = obj.key.lower() TARGET_KW = [x.lower() for x in TARGET_KW] IGNORE_KW = [x.lower() for x in IGNORE_KW] # filename must have all target kw's for targetkw in TARGET_KW: if filename.find(targetkw.lower()) != -1: matchcount += 1 # filename will ignore for any ignore kw match(es) if matchcount == len(TARGET_KW) and len(IGNORE_KW) > 0: for ignorekw in IGNORE_KW: if filename.find(ignorekw.lower()) != -1: ignoreflag = 1 # process only if no ignore kw triggered and ALL target kw matched if ignoreflag == 0 and matchcount == len(TARGET_KW): filelst.append(obj.key) return filelst # aws download function, file name is kept the same unless specified def downloadObjects(bucket, file, destfilename = None, path = None): if destfilename is None: destfilename = file.split('/')[-1] if path is not None: destfilename += path try: S3.Bucket(bucket).download_file(file, destfilename) except botocore.exceptions.ClientError as e: if e.response['Error']['Code'] == "404": print("The object does not exist.") else: raise