Python Methods – List All Files in a Directory

This tutorial provides several ways in Python to list all files in a directory such as os.walker, os.listdir, and glob along with examples. You may need such techniques, especially in Selenium Python automation or working with configuration/log files.

Python comes with the default OS module that enables several functions to interact with the file system. As mentioned above, it has a walk() method which lists all files inside a directory. Besides, it has another function listdir() that does find files on the specified path.

Similarly, Python’s Glob module has a glob() method that checks for the specified files in the current directory. Let’s now look at these functions in more detail and with examples.

Python List All Files in a Directory

Here, we are demonstrating functions that help traverse the file system and search for the files present.

Os.walk() method

It gathers the file names present in a directory by traversing the dir in either top-down or bottom-up. It returns a tuple of the following three:

• Root: Gets only the folders from the input.
• Dirs: Gets sub-directories from the root.
• Files: Gets all files from the given root and directories.

Find all text files in dirs and subdirs

Below is the sample Python code printing all files in given directories and sub-directories.

import os

location = 'c:/test/temp/'
files_in_dir = []

# r=>root, d=>directories, f=>files
for r, d, f in os.walk(location):
   for item in f:
      if '.txt' in item:
         files_in_dir.append(os.path.join(r, item))

for item in files_in_dir:
   print("file in dir: ", item)

After execution, the following is the result:

c:/test/temp/notes/readme.txt
c:/test/temp/release/artifact_list.txt
c:/test/temp/dist/doc/readme.txt
c:/test/temp/dist/samples/sample.txt

List all dirs under given dirs and subdirs

Check the below Python code to find and print all dirs under the given dir/subdir.

import os

location = 'c:/test/temp/'

dirs_in_dir = []

# r=>root, d=>directories, f=>files
for r, d, f in os.walk(location):
   for item in d:
      if '.txt' in item:
         dirs_in_dir.append(os.path.join(r, item))

for item in dirs_in_dir:
   print("Dirs under dir: ", item)

After execution, the following is the result:

c:/test/temp/notes/
c:/test/temp/release/
c:/test/temp/dist/
c:/test/temp/dist/doc/
c:/test/temp/dist/samples/

Glob.glob() method

Many times, we have to iterate over a list of files in a directory having names matching a pattern. In such a case, the glob module helps capture the list of files in a given directory with a particular extension.

glob() function

This function fetches a list of files filtered based on the given pattern in the pathname. We can take a pathname which is absolute as well as relative. The wild cards such as * and ? are also allowed symbols.

Another parameter, recursive is off (false) by default. If its value is True, then this function searches inside all subdirectories of the current directory and find files having the desired pattern

List all files in the current directory having “.py” extension

For example – The following code lists all files in the current directory having “.py” extension.

import glob

location = 'c:/test/temp/'

fileset = [file for file in glob.glob(location + "**/*.py", recursive=True)]

for file in fileset:
    print(file)

After execution, the following is the result:

c:/test/temp/notes/get_sample.py
c:/test/temp/release/test1.py
c:/test/temp/dist/doc/core.py
c:/test/temp/dist/samples/first_sample.py

Read about Python glob in more detail.

Get all dirs in a specified dir and subdirs

import glob

location = 'c:/test/temp/'

folderset = [folder for folder in glob.glob(location + "**/", recursive=True)]

for folder in folderset:
    print(folder)

After running the above code, the following is the result:

c:/test/temp/notes/
c:/test/temp/release/
c:/test/temp/dist/
c:/test/temp/dist/doc/
c:/test/temp/dist/samples/

Os.listdir() method to list text files

It gives a list including the names of the files in the directory specified in the location (path). The list happens to be in random order. It excludes the ‘.’ and ‘..’ if they are available in the input folder.

import os

location = 'c:/test/temp/'

for file in os.listdir(location):
    if file.endswith(".txt"):
        print(os.path.join(location, file))

After execution, the following is the result:

c:/test/temp/notes/readme.txt
c:/test/temp/release/artifact_list.txt
c:/test/temp/dist/doc/readme.txt
c:/test/temp/dist/samples/sample.txt

To learn Python in a step by step manner, read this Python tutorial.