de.lmu.ifi.dbs.elki.persistent
Class OnDiskArray

java.lang.Object
  de.lmu.ifi.dbs.elki.persistent.OnDiskArray

All Implemented Interfaces:

Serializable

public class OnDiskArray
extends Object
implements Serializable

On Disc Array storage for records of a given size. This can be used to implement various fixed size record-based data structures. The file format is designed to have a fixed-size header followed by the actual data.

See Also:

Serialized Form

Field Summary

private RandomAccessFile	file Random Access File object.
private File	filename File name.
private static int	HEADER_POS_SIZE Position of file size (in records)
private int	headersize Size of the header in the file.
private static int	INTERNAL_HEADER_SIZE Size of the classes header size.
private FileLock	lock Lock for the file that will be kept while writing.
protected int	magic Magic number used to identify files
private MappedByteBuffer	map
private int	numrecs Number of records in the file.
private int	recordsize Size of the records in the file.
private static long	serialVersionUID Serial version.
private boolean	writable Writable or read-only object

Constructor Summary

OnDiskArray(File filename, int magicseed, int extraheadersize, boolean writable)
Constructor to open an existing file.

OnDiskArray(File filename, int magicseed, int extraheadersize, int recordsize, boolean writable)
Constructor to open an existing file.

OnDiskArray(File filename, int magicseed, int extraheadersize, int recordsize, int initialsize)
Constructor to write a new file.

Method Summary

void	close() Explicitly close the file.
void	ensureSize(int size) Ensure that the file can fit the given number of records.
ByteBuffer	getExtraHeader() Read the extra header data.
protected int	getExtraHeaderSize() Return the size of the extra header.
File	getFilename() Get the file name.
int	getNumRecords() Get number of records in file.
ByteBuffer	getRecordBuffer(int index) Get a record buffer
protected int	getRecordsize() Get the size of a single record.
private long	indexToFileposition(long index) Compute file position from index number
boolean	isWritable() Check if the file is writable.
private void	mapArray() (Re-) map the data array.
static int	mixMagic(int magic1, int magic2) Mix two magic numbers into one, to obtain a combined magic.
void	resizeFile(int newsize) Resize file to the intended size
private void	validateHeader(boolean validateRecordSize) Validates the header and throws an IOException if the header is invalid.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serialVersionUID

private static final long serialVersionUID

Serial version. NOTE: Change this version whenever the file structure is changed in an incompatible way: This will modify the file magic, and thus prevent applications from reading incompatible files.

See Also:

Constant Field Values

magic

protected int magic

Magic number used to identify files

headersize

private int headersize

Size of the header in the file. Note that the internal header is four integers already.

recordsize

private int recordsize

Size of the records in the file.

numrecs

private int numrecs

Number of records in the file.

filename

private File filename

File name.

file

private final RandomAccessFile file

Random Access File object.

lock

private FileLock lock

Lock for the file that will be kept while writing.

writable

private boolean writable

Writable or read-only object

map

private MappedByteBuffer map

INTERNAL_HEADER_SIZE

private static final int INTERNAL_HEADER_SIZE

Size of the classes header size.

See Also:

Constant Field Values

HEADER_POS_SIZE

private static final int HEADER_POS_SIZE

Position of file size (in records)

See Also:

Constant Field Values

Constructor Detail

OnDiskArray

public OnDiskArray(File filename,
                   int magicseed,
                   int extraheadersize,
                   int recordsize,
                   int initialsize)
            throws IOException

Constructor to write a new file.

Parameters:

filename - File name to be opened.

magicseed - Magic number to derive real magic from.

extraheadersize - header size NOT including the internal header

recordsize - Record size

initialsize - Initial file size (in records)

Throws:

IOException - on IO errors

OnDiskArray

public OnDiskArray(File filename,
                   int magicseed,
                   int extraheadersize,
                   int recordsize,
                   boolean writable)
            throws IOException

Constructor to open an existing file. The provided record size must match the record size stored within the files header. If you don't know this size yet and/or need to access the extra header you should use the other constructor below

Parameters:

filename - File name to be opened.

magicseed - Magic number to derive real magic from.

extraheadersize - header size NOT including the internal header

recordsize - Record size

writable - flag to open the file writable

Throws:

IOException - on IO errors

OnDiskArray

public OnDiskArray(File filename,
                   int magicseed,
                   int extraheadersize,
                   boolean writable)
            throws IOException

Constructor to open an existing file. The record size is read from the file's header and can be obtained by getRecordsize()

Parameters:

filename - File name to be opened.

magicseed - Magic number to derive real magic from.

extraheadersize - header size NOT including the internal header

writable - flag to open the file writable

Throws:

IOException - on IO errors

Method Detail

mapArray

private void mapArray()
               throws IOException

(Re-) map the data array.

Throws:

IOException - on mapping error.

validateHeader

private void validateHeader(boolean validateRecordSize)
                     throws IOException

Validates the header and throws an IOException if the header is invalid. If validateRecordSize is set to true the record size must match exactly the stored record size within the files header, else the record size is read from the header and used.

Parameters:

validateRecordSize -

Throws:

IOException

mixMagic

public static final int mixMagic(int magic1,
                                 int magic2)

Mix two magic numbers into one, to obtain a combined magic. Note: mixMagic(a,b) != mixMagic(b,a) usually.

Parameters:

magic1 - Magic number to mix.

magic2 - Magic number to mix.

Returns:

Mixed magic number.

indexToFileposition

private long indexToFileposition(long index)

Compute file position from index number

Parameters:

index - Index offset

Returns:

file position

resizeFile

public void resizeFile(int newsize)
                throws IOException

Resize file to the intended size

Parameters:

newsize - New file size.

Throws:

IOException - on IO errors

getRecordBuffer

public ByteBuffer getRecordBuffer(int index)
                           throws IOException

Get a record buffer

Parameters:

index - Record index

Returns:

Byte buffer for the record

Throws:

IOException - on IO errors

getExtraHeaderSize

protected int getExtraHeaderSize()

Return the size of the extra header. Accessor.

Returns:

Extra header size

getExtraHeader

public ByteBuffer getExtraHeader()
                          throws IOException

Read the extra header data.

Returns:

additional header data

Throws:

IOException - on IO errors

getRecordsize

protected int getRecordsize()

Get the size of a single record.

Returns:

Record size.

getFilename

public File getFilename()

Get the file name.

Returns:

File name

isWritable

public boolean isWritable()

Check if the file is writable.

Returns:

true if the file is writable.

close

public void close()
           throws IOException

Explicitly close the file. Note: following operations will likely cause IOExceptions.

Throws:

IOException - on IO errors

getNumRecords

public int getNumRecords()

Get number of records in file.

Returns:

Number of records in the file.

ensureSize

public void ensureSize(int size)
                throws IOException

Ensure that the file can fit the given number of records.

Parameters:

size - Size

Throws:

IOException