Programming

Working with JSON Files in Python, with Examples

    Ini Arthur
    Share

    In this tutorial, we’ll learn how to read, write and parse JSON in Python with relevant examples. We’ll also explore popular modules in Python for working with JSON.

    JSON is a lightweight data interchange format. It’s a popular format used for transmitting and receiving data between a client and a server. However, its application and use go beyond transmitting data. Machines can easily generate and parse JSON data. The acronym JSON means JavaScript Object Notation, and, as the name rightly suggests, it’s a subset of the programming language JavaScript.

    JSON is a standardized data-interchange format, and it’s language independent. Virtually all programming languages support it in one way or another. It has the following structure:

    • It has an opening brace to the left and a closing brace to the right.
    • It has a collection of name/value pairs.
    • Each name is separated by a colon “:” from its value.
    • Each name/value pair is separated by a comma (,).

    Below is a snippet of a JSON object:

    {
    "name": "Chinedu Obi",
    "age": 24,
    "country": "Nigeria",
    "languages": [
        "Igbo",
        "English",
        "Yoruba"
    ],
    "marital status": "single",
    "employee": true,
    "experience": [
        {
            "title": "Frontend Engineering Intern",
            "company": "Andela"
        },
        {
            "title": "Frontend Engineer",
            "company": "Paystack"
        }
    ]
}

    We’ll assume that the above JSON is stored in a file named employee.json for the purposes of the later code examples.

    Types of JSON Data

    When working with JSON objects, Python converts JSON data types to their equivalents, and vice versa. The table below shows Python data types and their JSON equivalents.

    Python JSON Equivalent
    dict object
    list, tuple array
    str string
    int, float, long number
    True true
    False false
    None null

    The Difference Between the json and simplejson Modules in Python

    There are several modules for encoding and decoding JSON in Python. The two most popular modules are json and simplejson. The json module is a built-in package in the Python standard library, which means we can use it out of the box without having to install it.

    The simplejson module is an external Python module for encoding and decoding JSON. It’s an open-source package with backward compatibility for Python 2.5+ and Python 3.3+. It’s also fast, simple, correct and extensible.

    simplejson is updated more frequently, with more up-to-date optimization than json, making it faster. If you’re working with an older version of Python below 2.6 in a legacy project, then simplejson is your best bet. 

    For the purposes of this tutorial, we’ll stick with the json module.

    How to Read and Write JSON to a File in Python

    While programming in Python, we’ll often come across the JSON data format, and it’s important to know how to read or write JSON data and files. Prior knowledge of file handling in Python will come in handy here for reading and writing JSON to files.

    How to read JSON Files in Python

    Much like every other read operation in Python, the with statement can be used together with the json.load() method to read JSON files.

    See the code example below:

    import json

with open('employee.json', 'r', encoding='utf-8') as file_object:
    employee_dict = json.load(file_object)
    print(employee_dict)

    Here’s the output of the code above:

    {'name': 'Chinedu Obi', 'age': 24, 'country': 'Nigeria', 'languages': ['Igbo', 'English', 'Yoruba'], 'marital status': 'single', 'employee': True, 'experience': [{'title': 'Frontend Engineering Intern', 'company': 'Andela'}, {'title': 'Frontend Engineer', 'company': 'Paystack'}]}

    In the code above, we open the employee.json file in read mode. The json.load() method decodes the JSON data into a Python dictionary stored in the variable employee_dict.

    Writing JSON to a file in Python

    We can also perform a write operation with JSON data on files in Python. Just like in the read operation, we use the with statement alongside the json.dump() method to write JSON data to a file.

    Consider the code snippet below:

    import json

mother = {
    "name": "Asake Babatunde",
    "age": 28,
    "marital status": "Married",
    "children": ["Ayo", "Tolu", "Simi"],
    "staff": False,
    "next of kin": {"name": "Babatune Lanre", "relationship": "husband"},
}

with open("mother.json", "w", encoding="utf-8") as file_handle:
    json.dump(mother, file_handle, indent=4)

    Here, we create a Python dictionary, mother, which contains data about a fictional mother. We open mother.json in write mode. Since there’s no such file, one is created for us. The json.dump() method encodes the Python dictionary assigned to the mother variable to a JSON equivalent, which is written into the specified file. When the above code is executed, a mother.json file containing the JSON data will appear in the root of our folder.

    How to Convert a Python Dictionary to JSON (Serialization)

    Serialization is the process of converting Python objects — in most cases, a dictionary — to JSON formatted data or string. When serializing, Python types are encoded to a JSON equivalent. The json module provides two methods — json.dump() and json.dumps() — for serializing Python objects to JSON format.

    • json.dump() is used when writing JSON equivalent of Python objects to a file.
    • json.dumps() (with an “s”) is used when converting a Python object to a JSON-formatted string.

    Notice the syntax below:

    json.dump(obj, fp, indent)

    json.dumps(obj, indent)

    The json.dump() method has a parameter fp, while json.dumps() doesn’t have it.

    Some parameters explained:

    • obj: a Python object to be serialized to JSON format.
    • fp: a file pointer (object) with methods such as read() or write().
    • indent: a non-negative integer or string, to indicate indent level for pretty printing JSON data.

    Example: Converting a Python object to JSON format with json.dump()

    Let’s encode a Python object to equivalent JSON formatted data and write it to a file.

    First, we create a Python dictionary:

    import json

subject = {
    "name": "Biology",
    "teacher": {"name": "Nana Ama", "sex": "female"},
    "students_size": 24,
    "elective": True,
    "lesson days": ["Tuesday", "Friday"],
}

    Let’s encode our dictionary to JSON data and write to a file:

    with open('subject.json', 'w', encoding='utf-8') as file_handle:
    json.dump(subject, file_handle, indent=4)

    In the example above, we pass the dictionary, file pointer and indent arguments to the json.dump method. Below is the output of our code. When the code is executed, a subject.json file containing the expected JSON data is found in our project’s root folder:

    {
    "name": "Biology",
    "teacher": {
        "name": "Nana Ama",
        "sex": "female"
    },
    "students_size": 24,
    "elective": true,
    "lesson days": [
        "Tuesday",
        "Friday"
    ]
}

    Our output has a pretty printing output because we added an indent argument value of 4.

    Example: Converting a Python object to JSON format with json.dumps()

    In this example, we’ll encode a Python object to a JSON string. We created a subject dictionary before, so we can reuse it here.

    Let’s take an example with the json.dumps() method:

    json_data = json.dumps(subject, indent=4)
print(json_data)
print(type(json_data))

    Here’s the output of the code above:

    {
    "name": "Biology",
    "teacher": {
        "name": "Nana Ama",
        "sex": "female"
    },
    "students_size": 24,
    "elective": true,
    "lesson days": [
        "Tuesday",
        "Friday"
    ]
}
<class 'str'>

    As stated earlier, the json.dumps() method is used to convert Python objects to a JSON formatted string. We can see from the console that our JSON data is of type str.

    How to Convert JSON to a Python Dictionary (Deserialization)

    Deserialization of JSON is the process of decoding JSON objects to equivalent Python objects or Python types. We can convert JSON formatted data to Python objects using two methods — json.load() and json.loads() — which are provided by the json module.

    • json.load() is used when reading JSON formatted data from a file.
    • json.loads() (with an “s”) is used when parsing a JSON string to a Python dictionary.

    Notice the syntax below:

    json.load(fp)

    json.loads(s)

    json.dump() has a parameter fp, while json.dumps() has a parameter s. The other parameters remain the same.

    Some parameters explained:

    • fp: a file pointer (object) with methods such as read() or write().
    • s: a str, bytes or bytearray instance containing a JSON document.

    Example: Converting a JSON object to a Python object with json.load()

    Here are the contents of a new JSON file named students.json:

    [
    {
        "name": "Edidiong Ime",
        "subject": "Chemistry",
        "score": 91,
        "class": "SS 3"
    },
    {
        "name": "Jesse Umoh",
        "subject": "Chemistry",
        "score": 85,
        "class": "SS 3"
    },
    {
        "name": "Kingsley Chuks",
        "subject": "Chemistry",
        "score": 68,
        "class": "SHS 3"
    },
    {
        "name": "Martha Sam",
        "subject": "Chemistry",
        "score": 79,
        "class": "SHS 3"
    }
]

    In this example, we’ll decode JSON data from the students.json file to Python objects:

    import json

with open('students.json', 'r', encoding='utf-8') as file_handle:
    students_list = json.load(file_handle)
    for student in students_list:
        print(student)

    Here’s the output of the code above:

    {'name': 'Edidiong Ime', 'subject': 'Chemistry', 'score': 91, 'class': 'SS 3'}
{'name': 'Jesse Umoh', 'subject': 'Chemistry', 'score': 85, 'class': 'SS 3'}
{'name': 'Kingsley Chuks', 'subject': 'Chemistry', 'score': 68, 'class': 'SHS 3'}
{'name': 'Martha Sam', 'subject': 'Chemistry', 'score': 79, 'class': 'SHS 3'}

    In the code snippet above, a JSON file containing a list of students is being parsed. The JSON data from the file file_handle is passed to the json.load() method, which decodes it into a list of Python dictionaries. The list items are then printed to the console.

    Example: Converting a JSON object to a Python dictionary with json.loads()

    In this example, let’s decode JSON data from an API endpoint from JSONPlaceholder. The requests module should be installed before you proceed with this example:

    import json
import requests

response = requests.get("https://jsonplaceholder.typicode.com/comments/2")
comment = json.loads(response.text)

print(comment)

    Here’s the output of the code above:

    {'postId': 1, 'id': 2, 'name': 'quo vero reiciendis velit similique earum', 'email': 'Jayne_Kuhic@sydney.com', 'body': 'est natus enim nihil est dolore omnis voluptatem numquam\net omnis occaecati quod ullam at\nvoluptatem error expedita pariatur\nnihil sint nostrum voluptatem reiciendis et'}

    In the Python code above, we get a response from an endpoint that returns a JSON formatted string. We pass the response as an argument to the json.loads() method to decode it into a Python dictionary.

    Conclusion

    In modern web development, JSON is the de facto format for exchanging data between a server and web applications. These days, REST API endpoints return JSON-formatted data, so understanding how to work with JSON is important.

    Python has modules like json and simplejson for reading, writing and parsing JSON. The json module comes with the Python standard library, while simplejson is an external package and must be installed before being used.

    When building RESTful APIs in Python, or using external APIs in our projects, we’ll often need to serialize Python objects to JSON and deserialize them back to Python. The approaches demonstrated in this article are used by many popular projects. The steps are generally the same.

    The code from this tutorial can be found on GitHub.