Deleting multiple files

Suggest Edits

This example will show how you can delete multiple files with the API rate limit in mind. The optimal way to delete multiple files is via bulk API call which can delete up to 100 files.

Not optimized for rate limit

Fetch and delete files one by one using a loop.

for fn in source_file_names:
    f = api.files.query(project=src_project, names=[fn])[0]
    f.delete()

Optimized for rate limit

Fetch all files at once and then use a bulk API call to delete them in batches of 100 files or less.

def bulk_delete_files(files_to_delete, chunk_size=100):
    "Deletes files in bulk, 100 files per API call (max)"
     
    final_responses = []
     
    for i in range(0, len(files_to_delete), chunk_size):
         
        files = [f for f in files_to_delete[i:i + chunk_size]]
        responses = api.files.bulk_delete(files)
         
        for idx, r in enumerate(responses):
            if not r.valid:
                raise Exception(
                    '\n'.join([
                        str(r.error) + ": " + r.error.message,
                        r.error.more_info,
                        files[idx].name
                    ]))
         
        final_responses.extend(responses)
         
    return final_responses
 
files_to_delete = list(
    api.files.query(
        project=src_project,
        names=source_file_names,
        limit=100
    ).all())
 
responses = bulk_delete_files(files_to_delete)

Updated about 5 years ago