Python file handling is one of the essential topics for programmers and automation testers. As both of them needs to work with files either to write to a file or to read data from it. Also, if you are not already aware, I/O operations are the costliest operations where a program can stumble. Hence, you should be quite careful while implementing file handling for reporting or any other purpose. Optimizing a single file operation can help you produce an high-performing application or a robust solution for automated software testing.
Let’s take an example, say, you are going to create a big project in Python which contains a no. of workflows. Then, it’s inevitable for you not to create a log file. And you’ll also be doing both the read/write operations on the log file. Log files are a great tool to debug large programs. It’s always better to think about a scalable design from the beginning, as you won’t regret it later that you didn’t do it.
Here is a basic definition of file handling in Python, “File is a named location on the system storage which records data for later access. It enables persistent storage in a non-volatile memory i.e. Hard disk.“
In Python, file processing takes place in the following order.
- Open a file which returns a file handle.
- Use the handle to perform read or write action.
- Close the file handle.
Before you do a read or write operation to a file in Python, you need to open it first. And as the read/write transaction completes, you should close it to free the resources tied with the file.
In the next sections, we’ll touch upon all the Python file handling topics one by one. Since it’s an example-driven Python tutorial, so better you open a Python console to test-run the code.
First of all, let’s look at the summary of the Python file handling topics covered in this tutorial.
- Open a file in Python.
- Close a file in Python.
- Python Write operation.
- Python Read operation.
- File read positions in Python.
- Renaming and deleting files in Python.
- Python file object methods.
To read or write to a file, you need to open it first. To open a file in Python, use its built open() function. This function returns a file object i.e. a handle. You can use it to read or modify the file.
file object = open(file_name [, access_mode][, buffering])
Below are the parameter details.
<access_mode>- It’s an integer representing the file open mode e.g. read, write, append, etc. It’s an optional parameter. By default, it is set to read-only <r>. In this mode, we get data in text form after reading from the file. On the other hand, binary mode returns bytes. It’s preferable for accessing the non-text files like an image or the Exe files. See the table in the next section. It lists down the available access modes.
<buffering>- The default value is 0 which means buffering won’t happen. If the value is 1, then line buffering will take place while accessing the file. If it’s greater than 1, then the buffering action will run as per the buffer size. In the case of a negative value, the default behavior is considered.
<file_name>- It’s a string representing the name of the file you want to access.
|<r>||It opens a file in read-only mode while the file offset stays at the root.|
|<rb>||It opens a file in (binary + read-only) modes. And the offset remains at the root level.|
|<r+>||It opens the file in both (read + write) modes while the file offset is again at the root level.|
|<rb+>||It opens the file in (read + write + binary) modes. The file offset is again at the root level.|
|<w>||It allows write-level access to a file. If the file already exists, then it’ll get overwritten. It’ll create a new file if the same doesn’t exist.|
|<wb>||Use it to open a file for writing in binary format. Same behavior as for write-only mode.|
|<w+>||It opens a file in both (read + write) modes. Same behavior as for write-only mode.|
|<wb+>||It opens a file in (read + write + binary) modes. Same behavior as for write-only mode.|
|<a>||It opens the file in append mode. The offset goes to the end of the file. If the file doesn’t exist, then it gets created.|
|<ab>||It opens a file in (append + binary) modes. Same behavior as for append mode.|
|<a+>||It opens a file in (append + read) modes. Same behavior as for append mode.|
|<ab+>||It opens a file in (append + read + binary) modes. Same behavior as for append mode.|
When you call the Python open() function, it returns an object which is the file handle. Also, you should know that Python files have several linked attributes. And we can make use of the file handle to list the attributes of a file it belongs.
For more information on file attributes, please run down through the below table.
|<file.closed>||For a closed file, it returns true whereas false otherwise.|
|<file.mode>||It returns the access mode used to open a file.|
|<file.name>||It returns the name of a file.|
|<file.softspace>||It returns a boolean to suggest if a space char will get added before printing another value in the output of a <print> command.|
#Open a file in write and binary mode. fob = open("app.log", "wb") #Display file name. print "File name: ", fob.name #Display state of the file. print "File state: ", fob.closed #Print the opening mode. print "Opening mode: ", fob.mode #Output the softspace value. print "Softspace flag: ", fob.softspace
Python 2.7.10 [GCC 4.8.2] on Linux File name: app.log File state: False Opening mode: wb Softspace flag: 0
In Python 3.x, there is a clear difference between strings (text) and a byte (8-bits). It states that the char ‘a’ doesn’t represent the ASCII value 97 until you specify it like that. So, while you want to use a file in text mode, then better you mention the correct encoding type.
Also, Python stores a file in the form of bytes on the disk, so you need to decode them in strings before reading. And, similarly, encode them while writing texts to the file.
For a note, Python enables platform-dependent encoding by default. Hence, if you don’t change it, then it’s set to <cp1252> for Windows and <utf-8> for Linux.
Thus, the documentation says to quote the desired encoding while opening a file in Python. See the below Python code snippet.
f = open('app.log', mode = 'r', encoding = 'utf-8')
For a note, you should import the <io> module in Python 2.x to enable the encoding feature. Python 3.x does it implicitly.
It’s always the best practice to close a file when your work gets finished. However, Python runs a garbage collector to clean up the unused objects. But you must do it on your own rather leave it for the GC.
Python provides the <close()> method to close a file.
While closing a file, the system frees up all resources allocated to it. And it’s rather easy to achieve.
Please see from the below code snippets.
The most basic way is to call the Python close() method.
f = open("app.log",encoding = 'utf-8') # do file operations. f.close()
Say, if an exception occurs while performing some operations on the file. In such a case, the code exits without closing the file. So it’s better to put the code inside a <try-finally> block.
try: f = open('app.log', encoding = 'utf-8') # do file operations. finally: f.close()
So, even if there comes an exception, the above code will make sure your file gets closed properly.
Another way to close a file is by using the WITH clause. It ensures that the file gets closed when the block inside the WITH clause executes. The beauty of this method is that it doesn’t require to call the close() method explicitly.
with open('app.log', encoding = 'utf-8') as f: #do any file operation.
While you get ready for writing data to a file, first of all, open it using a mode (read/write/append). View the list of all available file modes here.
You can even do the same using the append mode. Also, if you’ve used the <w> mode, then it’ll erase the existing data from the file. So you must note this fact while you choose it.
Python provides the write() method to write a string or sequence of bytes to a file. This function returns a number which is the size of data written in a single Write call.
with open('app.log', 'w', encoding = 'utf-8') as f: #first line f.write('my first file\n') #second line f.write('This file\n') #third line f.write('contains three lines\n') with open('app.log', 'r', encoding = 'utf-8') as f: content = f.readlines() for line in content: print(line)
Python 3.5.1 [GCC 4.8.2] on Linux my first file This file contains three lines
To read data from a file, first of all, you need to open it in reading mode. Then, you can call anyone of the methods that Python provides for reading from a file.
Usually, you can use Python <read(size)> function to read the content of a file up to the size. If you don’t pass the size, then it’ll read the whole file.
with open('app.log', 'w', encoding = 'utf-8') as f: #first line f.write('my first file\n') #second line f.write('This file\n') #third line f.write('contains three lines\n') f = open('app.log', 'r', encoding = 'utf-8') print(f.read(10)) # read the first 10 data #'my first f' print(f.read(4)) # read the next 4 data #'ile\n' print(f.read()) # read in the rest till end of file #'This file\ncontains three lines\n' print(f.read()) # further reading returns empty sting #''
The <tell()> method gives you the current offset of the file pointer in a file.
The <seek(offset[, from])> method can help you change the position of a file pointer in a file.
The <offset> argument represents the size of the displacement while the <from> argument indicates the start point.
If the <from> value is 0, then the shift will start from the root level. If it’s value is 1, then the reference position will become the current position. Also, if it is 2, then the end of file would serve as the reference position.
with open('app.log', 'w', encoding = 'utf-8') as f: #first line f.write('It is my first file\n') #second line f.write('This file\n') #third line f.write('contains three lines\n') #Open a file f = open('app.log', 'r+') data = f.read(19); print('Read String is : ', data) #Check current position position = f.tell(); print('Current file position : ', position) #Reposition pointer at the beginning once again position = f.seek(0, 0); data = f.read(19); print('Again read String is : ', data) #Close the opened file f.close()
Python 3.5.1 [GCC 4.8.2] on Linux Read String is : It is my first file Current file position : 19 Again read String is : It is my first file
While you were using the <read/write> functions, you may also need to <rename/delete> a file in Python. So, there comes a <os> module in Python which brings the support of file <rename/delete> operations.
So, to continue, first of all, you should import the <os> module in your Python script.
The <rename()> method takes two arguments, the current filename, and the new filename.
Following is the example to rename an existing file <app.log> to <app1.log>.
import os #Rename a file from <app.log> to <app1.log> os.rename( "app.log", "app1.log" )
The <remove()> method deletes a file which it receives in the argument.
Following is the example to delete an existing file, the <app1.log>.
import os #Delete a file <app1.log> os.remove( "app1.log" )
So far, we’ve only shared with you a few of the functions that you can use for file handling in Python. But there is more to the story of Python file handling.
Since the Python’s open() method returns an object which we call as the file handle. So, Python adds a no. of functions that you can call using this object.
|<file.close()>||Close the file. You need to reopen it for further access.|
|<file.flush()>||Flush the internal buffer. It’s same as the <stdio>’s <fflush()> function.|
|<file.fileno()>||Returns an integer file descriptor.|
|<file.isatty()>||It returns true if file has a <tty> attached to it.|
|<file.next()>||Returns the next line from the last offset.|
|<file.read(size)>||Reads the given no. of bytes. It may read less if EOF is hit.|
|<file.readline(size)>||It’ll read an entire line (trailing with a new line char) from the file.|
|<file.readlines(size_hint)>||It calls the <readline()> to read until EOF. It returns a list of lines read from the file. If you pass <size_hint>, then it reads lines equalling the <size_hint> bytes.|
|<file.seek(offset[, from])>||Sets the file’s current position.|
|<file.tell()>||Returns the file’s current position.|
|<file.truncate(size)>||Truncates the file’s size. If the optional size argument is present, the file is truncated to (at most) that size.|
|<file.write(string)>||It writes a string to the file. And it doesn’t return any value.|
|<file.writelines(sequence)>||Writes a sequence of strings to the file. The sequence is possibly an iterable object producing strings, typically a list of strings.|
We wish the above Python file handling tutorial would have helped you in learning Python. If you want us to bring more such tutorials, then like and share this tutorial further.
Like we always say that it’s just the beginning, there is a long road ahead.
So keep climbing on the learning ladder.
All the Best,