Title: RandomAccessFile
Course: CPS 261: Advanced Java Concepts ~ DL & MM MASTER

RandomAccessFile

The Input/Output that we have been doing so far has all been sequential. We have classes that either read things sequentially or write things sequentially.

Two things that we sometimes would like to do:

- Move around within the file in a non-sequential way
- Be able to read and write a file with the same class. Imagine the concept of a database that accesses disk space such that it can read data or update data.

RandomAccessFile features:

- seek (long pos) - This moves our current access pointer to the "pos" location
- long getFilePointer() - Returns our current access pointer
- long length(); - returns the current length of the file
- void setLength(long newLength) - sets the current length of the file. This can be used to truncate or extend the file. Be aware that extending allocates space which has unknown values.
- writeInt(int), writeLong(long) , writeDouble(double), etc. - Similar methods to the ObjectOutputStream class
- writeUTF(String ) - For writing out a String
- int readInt(), long readLong(), double readDouble(), etc. - Similar methods to the ObjectInputStream class
- String readUTF() - For reading in a String
- Non readObject or writeObject - Unfortunately these methods are not present as they are in the Object Streams.

Example of how Random Access can be used:

The real world example I want to describe is the PDF (Portable Document Format) created by Adobe. When I was in industry, I once was required to dynamically create PDF documents from our C++ software. In the process of this effort I had a chance to get to know how PDF works. In my example below, I have created a simplified example that has some of the same characteristics of a PDF file.

First I would like to specify some of the characteristics of PDF:

- PDF can store a large document (perhaps a whole book) which describes the pages of the document. Because of the way PDF works, it is not necessary to load the whole document into memory to update it.
- Using PDF, you can conveniently pull out specific pages, or page ranges.
- PDF supports the ability to update pages within the document without requiring a complete rewrite of the whole document. For example, it is possible with PDF to update a page in the middle of the document.

RandomPeople example:

In order to show the concept of how PDF works without the complexities of how to represent page output, I have created a related problem where I am going to store multiple Person classes( my Person class will be real simple: name and age). In PDF, I might have a Page class, but in my example I will have a Person class instead.

In this simple example, I will demonstrate the following properties:

- Without reading the whole file in, I can conveniently read in any Person class desired. Although my pictures below only show 3 Person classes, imagine that the file could contain many Person classes (millions?).
- I also can update a Person class without rewriting the whole file.
- In fact, I will do this such that I can make a number of temporary updates, and then at the end I can choose to "commit" my updates. This is a concept that is used in Databases, where someone can make a number of updates and then at the end choose to commit or "rollback" all of the temporary changes. This is an important concept in fully functional databases like Oracle.
- In order accomplish the above actions, it will be necessary to leave "holes" or unused spaces within the file. To remove the "holes", I will then show you how to compress the file back to a file that has no unused spaces.

RandomPeople.java


import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.ArrayList;


public class RandomPeople {

    private ArrayList<Long> dirEntries = null;
    private RandomAccessFile ranFile= null;
    private String filename=null;
    private long nextWriteLocation = 0;

    private void check_if_file_open() throws IOException
    {
        if (ranFile == null)
            throw new IOException("Can't call this routine if RandomPeople has not been opened. ");
    }

    public long size() throws IOException
    {
        check_if_file_open();
        return dirEntries.size();
    }

    public int open(String filename) throws IOException
    {
        this.filename = filename;
        dirEntries = new ArrayList<Long>();
        ranFile = new RandomAccessFile(filename, "rw");
        long fileLength = ranFile.length();

        if (fileLength > 8)
        {
            long locationOfDirectory = ranFile.readLong();
            long numberOfDirEntries = ranFile.readLong();
            ranFile.seek(locationOfDirectory);

            for (int i=0; i < numberOfDirEntries; i++)
            {
                long dirEntry = ranFile.readLong();
                dirEntries.add(dirEntry);
            }
        }
        else
        {
            // New file that needs to be Prepped
            ranFile.writeLong(0); // No directory yet ... needs commit call
            ranFile.writeLong(0);
        }
        nextWriteLocation = ranFile.getFilePointer();
        return 0;
    }

    public void close() throws IOException
    {
        if (ranFile != null)
            ranFile.close();
    }
    public Person getPerson(int index) throws IOException
    {
        check_if_file_open();
        ranFile.seek(dirEntries.get(index));
        int age = ranFile.readInt();
        String name = ranFile.readUTF();
        return  new Person(name, age);
    }
    public void replacePerson(Person person, int index) throws IOException
    {
        check_if_file_open();
        long fileLocation = append(person);
        dirEntries.set(index, fileLocation);
    }
    public int addPerson(Person person) throws IOException
    {
        check_if_file_open();
        long fileLocation = append(person);
        int size = dirEntries.size();
        dirEntries.add(size, fileLocation);
        return size;
    }
    private long append(Person person) throws IOException
    {
        check_if_file_open();
        long location = nextWriteLocation;
        ranFile.seek(nextWriteLocation);
        ranFile.writeInt(person.age);
        ranFile.writeUTF(person.name);
        nextWriteLocation = ranFile.getFilePointer();
        return location; // This is where the Person was written
    }
    public void commit() throws IOException
    {
        check_if_file_open();
        long newDirLocation = nextWriteLocation;
        ranFile.seek(newDirLocation);
        for (int i=0; i < dirEntries.size(); i++)
        {
            ranFile.writeLong(dirEntries.get(i));
        }
        nextWriteLocation = ranFile.getFilePointer();

        ranFile.seek(0);
        ranFile.writeLong(newDirLocation);
        ranFile.writeLong(dirEntries.size());
    }


    public void compress() throws IOException
    {
        check_if_file_open();
        String tempFilename = "temp_"+filename;
        RandomAccessFile tempRanFile = new RandomAccessFile(tempFilename, "rw");
        ArrayList<Long> tempDirEntries = new ArrayList<Long>();

        tempRanFile.writeLong(0); // we will fill this in later with the location of the directory
        tempRanFile.writeLong(dirEntries.size()); // the number of directory entries

        // Copy Records of Person
        for (int i=0; i < dirEntries.size(); i++)
        {
            tempDirEntries.add(tempRanFile.length()); // remember the current location
            Person p = getPerson(i); // Read for current file
            tempRanFile.writeInt(p.age);
            tempRanFile.writeUTF(p.name);
        }
        // End of Person data
        long location_of_directory = tempRanFile.getFilePointer();

        // Write out the directory
        for (int i = 0; i < tempDirEntries.size(); i++)
        {
            tempRanFile.writeLong( tempDirEntries.get(i));
        }

        tempRanFile.seek(0);
        tempRanFile.writeLong(location_of_directory); // Final data is compressed and up to date in the temp file

        // Close out files
        ranFile.close();  // First close existing file
        tempRanFile.close(); // close temp file

        // tempRanFile is now complete.  Need to move it to the currently open file
        File ftemp = new File(tempFilename);
        File fcurr = new File(filename);

        fcurr.delete(); // delete the old non-compressed version
        ftemp.renameTo(fcurr); // rename the tempRanFile to the original filename.

        open(filename);  // re-open newly compressed file


    }



}



Person.java


public class Person {
    int age;
    String name;
    Person (String name, int age)
    {
        this.age = age;
        this.name = name;
    }
    public String toString()
    {
        return name + " : " + age;
    }

}


DemoRandomPeople.java


import java.io.IOException;
import java.util.Scanner;


public class DemoRandomPeople {


    public static void main(String[] args) {
        RandomPeople rp = new RandomPeople();
        Scanner keyboard = new Scanner(System.in);
        boolean continueFlag = true;

        try {
            rp.open("RandomPeople.dat");
        } catch (IOException e1) {
            // TODO Auto-generated catch block
            e1.printStackTrace();
            return;
        }

        while(continueFlag)
        {
            try
            {
                System.out.println("Cmds: [get index] [add name age] [replace index name age] [list] [commit] [compress][quit]");
                String cmd = keyboard.next();
                String name;
                int index;
                int age;
                System.out.println("Command =" + cmd);

                switch(cmd)
                {
                case "get":
                    index = keyboard.nextInt();
                    Person p = rp.getPerson(index);
                    System.out.println("Person = "+ p);
                    break;

                case "add":
                    name = keyboard.next();
                    age = keyboard.nextInt();
                    rp.addPerson(new Person(name, age));
                    break;
                case "replace":
                    index = keyboard.nextInt();
                    name = keyboard.next();
                    age = keyboard.nextInt();
                    rp.replacePerson(new Person(name, age), index);
                    break;
                case "list":
                    long size = rp.size();
                    for (int i =0; i < size; i++)
                    {
                        Person p2 = rp.getPerson(i);
                        System.out.println("Person("+i+") = "+ p2);
                    }
                    break;
                case "commit":
                    rp.commit();
                    break;
                case "compress":
                    rp.compress();
                    break;
                case "quit":
                    continueFlag = false;
                    break;
                }
            }

            catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace(System.out);
            }
            catch (RuntimeException e){
                e.printStackTrace(System.out);
            }
            keyboard.nextLine();
        }


        try {
            rp.close();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } // End of while(continueFlag) loop

        System.out.println("Bye");

    }



}



Some notes for Clem

The following is NOT on the video. I wish RandomAccessFile had readObject and writeObject. I did a google search and came up with some notes on how I might in the future provide my own readObject and writeObject method. You should probably just ignore the following because it is a raw copy from the web that I wanted to save.


package com.example;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

public class Serializer {
    public static byte[] serialize(Object obj) throws IOException {
        ByteArrayOutputStream b = new ByteArrayOutputStream();
        ObjectOutputStream o = new ObjectOutputStream(b);
        o.writeObject(obj);
        return b.toByteArray();
    }

    public static Object deserialize(byte[] bytes) throws IOException, ClassNotFoundException {
        ByteArrayInputStream b = new ByteArrayInputStream(bytes);
        ObjectInputStream o = new ObjectInputStream(b);
        return o.readObject();
    }
}




There are several improvements to this that can be done. Not in the least the fact that you can only read/write one object per byte array, which might or might not be what you want.

Note that "Only objects that support the java.io.Serializable interface can be written to streams" (see java.io.ObjectOutputStream). Since you might run into it, the continuous allocation and resizing of the java.io.ByteArrayOutputStream might turn out to be quite the bottle neck.

Depending on your threading model you might want to consider reusing some of the objects. For serialization of objects that do not implement the Serializable interface you either need to write your own serializer, for example using the read*/write* methods of java.io.DataOutputStream and the get*/put* methods of java.nio.ByteBuffer perhaps together with reflection, or pull in a third party dependency. This site has a list and performance comparison of some serialization frameworks. Looking at the APIs it seems Kryo might fit what you need.

Last Updated: July 30, 2014 10:04 PM