What Is Fast Way to Check 'Isdir' in Python?
In My Project, My Program Check Very Many Dir and Files. I Use Python 3.8. First, I Try This. Lower Is My Code Snippet. If '.' Not in Filename: # Dir I Know...
In my project, my program check very many dir and files.
I use python 3.8.
first, i try this. lower is my code snippet.
if '.' not in filename: # dir
I know, it's not right. directory name can include '.'!
I try this.
if os.path.isdir(os.path.join(path_dir, filename)): # dir
It's right. but, upper code is terribly slow!
first code use 0.48 sec. but second use 17 sec...
I change [os.path.join] -> [path_dir + '\' + filename]. It takes 15 sec. it's too slow, too.
I know os.path.isdir is problem. lower is test.
@logging_time
def test_func1(dir_name, file_name):
full_path = os.path.join(dir_name, file_name)
for i in range(100000):
if os.path.isdir(full_path):
res = 'a'
@logging_time
def test_func2(dir_name, file_name):
for i in range(100000):
if '.' not in file_name:
res = 'a'
test_func2 use just 0.003 sec, but test_func1 use 3 sec!
What is the fast way to check isDir?
Target OS is windows 10. I don't need cross-flatform. just window.
2 Answers
os.path.isdir seems to be the fastest implementation:
from pathlib import Path
import os
import win32api
import glob
import timeit
def os_isDir(files):
return [os.path.isdir(fullPath) for fullPath in files]
def os_NOT_isFile(files):
return [not os.path.isfile(fullPath) for fullPath in files]
def pathlib_isDir(files):
return [Path(fullPath).is_dir() for fullPath in files]
def winapi_GetFileAttributesA(files):
return [win32api.GetFileAttributes(fullPath) == 16 for fullPath in files]
files = glob.glob('C:\\Windows\\System32\\*')
for func in ("os_isDir", "os_NOT_isFile", "pathlib_isDir", "winapi_GetFileAttributesA"):
t1 = timeit.Timer(f"{func}({files})", f"from __main__ import {func}")
print(f"Evaluation {len(files)} paths using {func} took: {t1.timeit(number=5)} seconds\n")
Out (on a Win7 VM):
Evaluation 3294 paths using os_isDir took: 2.1250754 seconds
Evaluation 3294 paths using os_NOT_isFile took: 2.7074766999999995 seconds
Evaluation 3294 paths using pathlib_isDir took: 3.4624453000000006 seconds
Evaluation 3294 paths using winapi_GetFileAttributesA took: 2.2789655999999994 seconds
Perhaps the fastest way to check if something is a directory is to not have to check at all. If you wish to iterate all files in a directory tree (you haven't actually specified whether this is the case), you should consider using os.walk:
os.walk(top, topdown=True, onerror=None, followlinks=False)
Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at directory top (including top itself), it yields a 3-tuple (dirpath, dirnames, filenames).
The 3rd element of the tuple (filenames) is guaranteed to not be a directory. This is how you would print all file names in a directory tree:
import os
def walk_directory(directory_name):
for dirpath, dirnames, filenames in os.walk(directory_name):
for filename in filenames:
fullpathname = os.path.join(dirpath, filename)
print(fullpathname)
Otherwise, you should be using an isDir method call. For example, if you just wanted to iterate all the files in a single directory, then one way is as follows:
from pathlib import Path
def list_directory(directory_name):
# non-recursive
for p in Path(directory_name).glob('*'):
if not p.is_dir():
#filename = str(p) # to convert Path to a string
print(p)
Chloe Bennett explores the intersection of pop culture, streaming entertainment, digital trends, and contemporary lifestyle. Her weekly commentary reaches thousands of culture enthusiasts.
