de.lmu.ifi.dbs.elki.persistent
Class OnDiskArray

java.lang.Object
  extended by de.lmu.ifi.dbs.elki.persistent.OnDiskArray
All Implemented Interfaces:
Serializable

public class OnDiskArray
extends Object
implements Serializable

On Disc Array storage for records of a given size. This can be used to implement various fixed size record-based data structures. The file format is designed to have a fixed-size header followed by the actual data.

See Also:
Serialized Form

Field Summary
private  RandomAccessFile file
          Random Access File object.
private  File filename
          File name.
private static int HEADER_POS_SIZE
          Position of file size (in records)
private  int headersize
          Size of the header in the file.
private static int INTERNAL_HEADER_SIZE
          Size of the classes header size.
private  FileLock lock
          Lock for the file that will be kept while writing.
protected  int magic
          Magic number used to identify files
private  MappedByteBuffer map
           
private  int numrecs
          Number of records in the file.
private  int recordsize
          Size of the records in the file.
private static long serialVersionUID
          Serial version.
private  boolean writable
          Writable or read-only object
 
Constructor Summary
OnDiskArray(File filename, int magicseed, int extraheadersize, boolean writable)
          Constructor to open an existing file.
OnDiskArray(File filename, int magicseed, int extraheadersize, int recordsize, boolean writable)
          Constructor to open an existing file.
OnDiskArray(File filename, int magicseed, int extraheadersize, int recordsize, int initialsize)
          Constructor to write a new file.
 
Method Summary
 void close()
          Explicitly close the file.
 void ensureSize(int size)
          Ensure that the file can fit the given number of records.
 ByteBuffer getExtraHeader()
          Read the extra header data.
protected  int getExtraHeaderSize()
          Return the size of the extra header.
 File getFilename()
          Get the file name.
 int getNumRecords()
          Get number of records in file.
 ByteBuffer getRecordBuffer(int index)
          Get a record buffer
protected  int getRecordsize()
          Get the size of a single record.
private  long indexToFileposition(long index)
          Compute file position from index number
 boolean isWritable()
          Check if the file is writable.
private  void mapArray()
          (Re-) map the data array.
static int mixMagic(int magic1, int magic2)
          Mix two magic numbers into one, to obtain a combined magic.
 void resizeFile(int newsize)
          Resize file to the intended size
private  void validateHeader(boolean validateRecordSize)
          Validates the header and throws an IOException if the header is invalid.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

serialVersionUID

private static final long serialVersionUID
Serial version. NOTE: Change this version whenever the file structure is changed in an incompatible way: This will modify the file magic, and thus prevent applications from reading incompatible files.

See Also:
Constant Field Values

magic

protected int magic
Magic number used to identify files


headersize

private int headersize
Size of the header in the file. Note that the internal header is four integers already.


recordsize

private int recordsize
Size of the records in the file.


numrecs

private int numrecs
Number of records in the file.


filename

private File filename
File name.


file

private final RandomAccessFile file
Random Access File object.


lock

private FileLock lock
Lock for the file that will be kept while writing.


writable

private boolean writable
Writable or read-only object


map

private MappedByteBuffer map

INTERNAL_HEADER_SIZE

private static final int INTERNAL_HEADER_SIZE
Size of the classes header size.

See Also:
Constant Field Values

HEADER_POS_SIZE

private static final int HEADER_POS_SIZE
Position of file size (in records)

See Also:
Constant Field Values
Constructor Detail

OnDiskArray

public OnDiskArray(File filename,
                   int magicseed,
                   int extraheadersize,
                   int recordsize,
                   int initialsize)
            throws IOException
Constructor to write a new file.

Parameters:
filename - File name to be opened.
magicseed - Magic number to derive real magic from.
extraheadersize - header size NOT including the internal header
recordsize - Record size
initialsize - Initial file size (in records)
Throws:
IOException - on IO errors

OnDiskArray

public OnDiskArray(File filename,
                   int magicseed,
                   int extraheadersize,
                   int recordsize,
                   boolean writable)
            throws IOException
Constructor to open an existing file. The provided record size must match the record size stored within the files header. If you don't know this size yet and/or need to access the extra header you should use the other constructor below

Parameters:
filename - File name to be opened.
magicseed - Magic number to derive real magic from.
extraheadersize - header size NOT including the internal header
recordsize - Record size
writable - flag to open the file writable
Throws:
IOException - on IO errors

OnDiskArray

public OnDiskArray(File filename,
                   int magicseed,
                   int extraheadersize,
                   boolean writable)
            throws IOException
Constructor to open an existing file. The record size is read from the file's header and can be obtained by getRecordsize()

Parameters:
filename - File name to be opened.
magicseed - Magic number to derive real magic from.
extraheadersize - header size NOT including the internal header
writable - flag to open the file writable
Throws:
IOException - on IO errors
Method Detail

mapArray

private void mapArray()
               throws IOException
(Re-) map the data array.

Throws:
IOException - on mapping error.

validateHeader

private void validateHeader(boolean validateRecordSize)
                     throws IOException
Validates the header and throws an IOException if the header is invalid. If validateRecordSize is set to true the record size must match exactly the stored record size within the files header, else the record size is read from the header and used.

Parameters:
validateRecordSize -
Throws:
IOException

mixMagic

public static final int mixMagic(int magic1,
                                 int magic2)
Mix two magic numbers into one, to obtain a combined magic. Note: mixMagic(a,b) != mixMagic(b,a) usually.

Parameters:
magic1 - Magic number to mix.
magic2 - Magic number to mix.
Returns:
Mixed magic number.

indexToFileposition

private long indexToFileposition(long index)
Compute file position from index number

Parameters:
index - Index offset
Returns:
file position

resizeFile

public void resizeFile(int newsize)
                throws IOException
Resize file to the intended size

Parameters:
newsize - New file size.
Throws:
IOException - on IO errors

getRecordBuffer

public ByteBuffer getRecordBuffer(int index)
                           throws IOException
Get a record buffer

Parameters:
index - Record index
Returns:
Byte buffer for the record
Throws:
IOException - on IO errors

getExtraHeaderSize

protected int getExtraHeaderSize()
Return the size of the extra header. Accessor.

Returns:
Extra header size

getExtraHeader

public ByteBuffer getExtraHeader()
                          throws IOException
Read the extra header data.

Returns:
additional header data
Throws:
IOException - on IO errors

getRecordsize

protected int getRecordsize()
Get the size of a single record.

Returns:
Record size.

getFilename

public File getFilename()
Get the file name.

Returns:
File name

isWritable

public boolean isWritable()
Check if the file is writable.

Returns:
true if the file is writable.

close

public void close()
           throws IOException
Explicitly close the file. Note: following operations will likely cause IOExceptions.

Throws:
IOException - on IO errors

getNumRecords

public int getNumRecords()
Get number of records in file.

Returns:
Number of records in the file.

ensureSize

public void ensureSize(int size)
                throws IOException
Ensure that the file can fit the given number of records.

Parameters:
size - Size
Throws:
IOException

Release 0.4.0 (2011-09-20_1324)