Deleting multiple files
This example will show how you can delete multiple files with the API rate limit in mind. The optimal way to delete multiple files is via bulk API call which can delete up to 100 files.
Not optimized for rate limit
Fetch and delete files one by one using a loop.
for fn in source_file_names: f = api.files.query(project=src_project, names=[fn]) f.delete()
Optimized for rate limit
Fetch all files at once and then use a bulk API call to delete them in batches of 100 files or less.
def bulk_delete_files(files_to_delete, chunk_size=100): "Deletes files in bulk, 100 files per API call (max)" final_responses =  for i in range(0, len(files_to_delete), chunk_size): files = [f for f in files_to_delete[i:i + chunk_size]] responses = api.files.bulk_delete(files) for idx, r in enumerate(responses): if not r.valid: raise Exception( '\n'.join([ str(r.error) + ": " + r.error.message, r.error.more_info, files[idx].name ])) final_responses.extend(responses) return final_responses files_to_delete = list( api.files.query( project=src_project, names=source_file_names, limit=100 ).all()) responses = bulk_delete_files(files_to_delete)
Updated over 3 years ago