OnDiskArray (ELKI: Environment for DeveLoping KDD-Applications Supported by Index-Structures)

java.lang.Object
- de.lmu.ifi.dbs.elki.persistent.OnDiskArray

All Implemented Interfaces:

java.lang.AutoCloseable
```
public class OnDiskArray
extends java.lang.Object
implements java.lang.AutoCloseable
```
On Disc Array storage for records of a given size. This can be used to implement various fixed size record-based data structures. The file format is designed to have a fixed-size header followed by the actual data.

Since:

0.2

Author:

Erich Schubert

Field Summary

Fields
Modifier and Type	Field and Description
`private java.io.RandomAccessFile`	`file` Random Access File object.
`private java.io.File`	`filename` File name.
`private static int`	`HEADER_POS_SIZE` Position of file size (in records).
`private int`	`headersize` Size of the header in the file.
`private static int`	`INTERNAL_HEADER_SIZE` Size of the classes header size.
`private java.nio.channels.FileLock`	`lock` Lock for the file that will be kept while writing.
`protected int`	`magic` Magic number used to identify files.
`private java.nio.MappedByteBuffer`	`map` The memory mapped buffer.
`private int`	`numrecs` Number of records in the file.
`private int`	`recordsize` Size of the records in the file.
`private static long`	`serialVersionUID` Serial version.
`private boolean`	`writable` Writable or read-only object.

Constructor Summary

Constructors
Constructor and Description
`OnDiskArray(java.io.File filename, int magicseed, int extraheadersize, boolean writable)` Constructor to open an existing file.
`OnDiskArray(java.io.File filename, int magicseed, int extraheadersize, int recordsize, boolean writable)` Constructor to open an existing file.
`OnDiskArray(java.io.File filename, int magicseed, int extraheadersize, int recordsize, int initialsize)` Constructor to write a new file.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`close()` Explicitly close the file.
`void`	`ensureSize(int size)` Ensure that the file can fit the given number of records.
`java.nio.ByteBuffer`	`getExtraHeader()` Read the extra header data.
`protected int`	`getExtraHeaderSize()` Return the size of the extra header.
`java.io.File`	`getFilename()` Get the file name.
`int`	`getNumRecords()` Get number of records in file.
`java.nio.ByteBuffer`	`getRecordBuffer(int index)` Get a record buffer
`protected int`	`getRecordsize()` Get the size of a single record.
`private long`	`indexToFileposition(long index)` Compute file position from index number
`boolean`	`isWritable()` Check if the file is writable.
`private void`	`mapArray()` (Re-) map the data array.
`static int`	`mixMagic(int magic1, int magic2)` Mix two magic numbers into one, to obtain a combined magic.
`void`	`resizeFile(int newsize)` Resize file to the intended size
`private void`	`validateHeader(boolean validateRecordSize)` Validates the header and throws an IOException if the header is invalid.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - serialVersionUID
```
private static final long serialVersionUID
```
    Serial version. NOTE: Change this version whenever the file structure is changed in an incompatible way: This will modify the file magic, and thus prevent applications from reading incompatible files.
    
    See Also:
    
    Constant Field Values
  - magic
```
protected int magic
```
    Magic number used to identify files.
  - headersize
```
private int headersize
```
    Size of the header in the file. Note that the internal header is four integers already.
  - recordsize
```
private int recordsize
```
    Size of the records in the file.
  - numrecs
```
private int numrecs
```
    Number of records in the file.
  - filename
```
private java.io.File filename
```
    File name.
  - file
```
private final java.io.RandomAccessFile file
```
    Random Access File object.
  - lock
```
private java.nio.channels.FileLock lock
```
    Lock for the file that will be kept while writing.
  - writable
```
private boolean writable
```
    Writable or read-only object.
  - map
```
private java.nio.MappedByteBuffer map
```
    The memory mapped buffer.
  - INTERNAL_HEADER_SIZE
```
private static final int INTERNAL_HEADER_SIZE
```
    Size of the classes header size.
    
    See Also:
    
    Constant Field Values
  - HEADER_POS_SIZE
```
private static final int HEADER_POS_SIZE
```
    Position of file size (in records).
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - OnDiskArray
```
public OnDiskArray(java.io.File filename,
                   int magicseed,
                   int extraheadersize,
                   int recordsize,
                   int initialsize)
            throws java.io.IOException
```
    Constructor to write a new file.
    
    Parameters:
    
    filename - File name to be opened.
    
    magicseed - Magic number to derive real magic from.
    
    extraheadersize - header size NOT including the internal header
    
    recordsize - Record size
    
    initialsize - Initial file size (in records)
    
    Throws:
    
    java.io.IOException - on IO errors
  - OnDiskArray
```
public OnDiskArray(java.io.File filename,
                   int magicseed,
                   int extraheadersize,
                   int recordsize,
                   boolean writable)
            throws java.io.IOException
```
    Constructor to open an existing file. The provided record size must match the record size stored within the files header. If you don't know this size yet and/or need to access the extra header you should use the other constructor below
    
    Parameters:
    
    filename - File name to be opened.
    
    magicseed - Magic number to derive real magic from.
    
    extraheadersize - header size NOT including the internal header
    
    recordsize - Record size
    
    writable - flag to open the file writable
    
    Throws:
    
    java.io.IOException - on IO errors
  - OnDiskArray
```
public OnDiskArray(java.io.File filename,
                   int magicseed,
                   int extraheadersize,
                   boolean writable)
            throws java.io.IOException
```
    Constructor to open an existing file. The record size is read from the file's header and can be obtained by getRecordsize()
    
    Parameters:
    
    filename - File name to be opened.
    
    magicseed - Magic number to derive real magic from.
    
    extraheadersize - header size NOT including the internal header
    
    writable - flag to open the file writable
    
    Throws:
    
    java.io.IOException - on IO errors
- Method Detail
  - mapArray
```
private void mapArray()
               throws java.io.IOException
```
    (Re-) map the data array.
    
    Throws:
    
    java.io.IOException - on mapping error.
  - validateHeader
```
private void validateHeader(boolean validateRecordSize)
                     throws java.io.IOException
```
    Validates the header and throws an IOException if the header is invalid. If validateRecordSize is set to true the record size must match exactly the stored record size within the files header, else the record size is read from the header and used.
    
    Parameters:
    
    validateRecordSize -
    
    Throws:
    
    java.io.IOException
  - mixMagic
```
public static final int mixMagic(int magic1,
                                 int magic2)
```
    Mix two magic numbers into one, to obtain a combined magic. Note: mixMagic(a,b) != mixMagic(b,a) usually.
    
    Parameters:
    
    magic1 - Magic number to mix.
    
    magic2 - Magic number to mix.
    
    Returns:
    
    Mixed magic number.
  - indexToFileposition
```
private long indexToFileposition(long index)
```
    Compute file position from index number
    
    Parameters:
    
    index - Index offset
    
    Returns:
    
    file position
  - resizeFile
```
public void resizeFile(int newsize)
                throws java.io.IOException
```
    Resize file to the intended size
    
    Parameters:
    
    newsize - New file size.
    
    Throws:
    
    java.io.IOException - on IO errors
  - getRecordBuffer
```
public java.nio.ByteBuffer getRecordBuffer(int index)
                                    throws java.io.IOException
```
    Get a record buffer
    
    Parameters:
    
    index - Record index
    
    Returns:
    
    Byte buffer for the record
    
    Throws:
    
    java.io.IOException - on IO errors
  - getExtraHeaderSize
```
protected int getExtraHeaderSize()
```
    Return the size of the extra header. Accessor.
    
    Returns:
    
    Extra header size
  - getExtraHeader
```
public java.nio.ByteBuffer getExtraHeader()
                                   throws java.io.IOException
```
    Read the extra header data.
    
    Returns:
    
    additional header data
    
    Throws:
    
    java.io.IOException - on IO errors
  - getRecordsize
```
protected int getRecordsize()
```
    Get the size of a single record.
    
    Returns:
    
    Record size.
  - getFilename
```
public java.io.File getFilename()
```
    Get the file name.
    
    Returns:
    
    File name
  - isWritable
```
public boolean isWritable()
```
    Check if the file is writable.
    
    Returns:
    
    true if the file is writable.
  - close
```
public void close()
           throws java.io.IOException
```
    Explicitly close the file. Note: following operations will likely cause IOExceptions.
    
    Specified by:
    
    close in interface java.lang.AutoCloseable
    
    Throws:
    
    java.io.IOException - on IO errors
  - getNumRecords
```
public int getNumRecords()
```
    Get number of records in file.
    
    Returns:
    
    Number of records in the file.
  - ensureSize
```
public void ensureSize(int size)
                throws java.io.IOException
```
    Ensure that the file can fit the given number of records.
    
    Parameters:
    
    size - Size
    
    Throws:
    
    java.io.IOException

Class OnDiskArray

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

serialVersionUID

magic

headersize

recordsize

numrecs

filename

file

lock

writable

map

INTERNAL_HEADER_SIZE

HEADER_POS_SIZE

Constructor Detail

OnDiskArray

OnDiskArray

OnDiskArray

Method Detail

mapArray

validateHeader

mixMagic

indexToFileposition

resizeFile

getRecordBuffer

getExtraHeaderSize

getExtraHeader

getRecordsize

getFilename

isWritable

close

getNumRecords

ensureSize