This repository was archived by the owner on Sep 27, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 69
Лебедев Алексей, 393, Parallel #584
Open
AlekseyLebedev
wants to merge
38
commits into
dkomanov:master
Choose a base branch
from
AlekseyLebedev:DevParallel
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
38 commits
Select commit
Hold shift + click to select a range
e836cee
Begin to do Storeable task. Something (can't build now, nothing works…
AlekseyLebedev 4b0e2b6
JUnit task v1.2.1 (Make some refactoring, to make it possible to do 5…
AlekseyLebedev 03a98aa
Json in temp folder
AlekseyLebedev 62b421d
Developing storeable
AlekseyLebedev c23bddd
Move JSON to Storeable folder
AlekseyLebedev 417b4a5
Add JSON
AlekseyLebedev 31d4a3a
Some cleanup
AlekseyLebedev 3042b88
Something
AlekseyLebedev e94304e
Add offset number to JSON
AlekseyLebedev c1defc4
Developing (commit to chenge branch)
AlekseyLebedev be23557
Add offset number to JSON
AlekseyLebedev a5928cb
Add one test
AlekseyLebedev 760b884
Add one test
AlekseyLebedev e5bbfe4
Rename to remove spelling mistake
AlekseyLebedev 025d749
JSON renaming in tests
AlekseyLebedev a731642
Storeable Table
AlekseyLebedev d44245f
JSON renaming in tests
AlekseyLebedev 9ff9c65
Rename to remove spelling mistake
AlekseyLebedev 5ca9f54
JUnit v1.3 (add final to variable)ш
AlekseyLebedev 70736ab
Strings Tests adapted for storeable (1 fails)
AlekseyLebedev aa28b2e
Fix test bug
AlekseyLebedev 59d0e87
Interpreter and tests for command
AlekseyLebedev 7b8ccff
Old interpreter tests
AlekseyLebedev cced64d
Interpreter fixes and tests, checkstyle
AlekseyLebedev dc9fcc7
Tests and fixes
AlekseyLebedev 0a37cb6
A few tests added and null-bug fix
AlekseyLebedev 81827a1
Fix bug in table with 0 columns
AlekseyLebedev 30620d6
Storeable v1.0 (checkstyle)
AlekseyLebedev e2ec6df
Merge branch 'DevJSON'
AlekseyLebedev ea75634
Merge branch 'master' into DevStorable
AlekseyLebedev 349b4ed
Storeable v2.0 (fix problems with changed interfaces)
AlekseyLebedev 1ec28a2
Parallel table
AlekseyLebedev 756c942
Parallel tableProvider
AlekseyLebedev 64cc20d
Parallel working in table optimisations
AlekseyLebedev 1948c15
Tests for parllel work
AlekseyLebedev 9cd824d
Remove commented code
AlekseyLebedev c606b3c
checkstyle fix
AlekseyLebedev aaa94a3
Fix bug in test
AlekseyLebedev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
361 changes: 361 additions & 0 deletions
361
src/ru/fizteh/fivt/students/LebedevAleksey/storeable/Database.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,361 @@ | ||
| package ru.fizteh.fivt.students.LebedevAleksey.storeable; | ||
|
|
||
| import ru.fizteh.fivt.storage.structured.ColumnFormatException; | ||
| import ru.fizteh.fivt.storage.structured.Storeable; | ||
| import ru.fizteh.fivt.storage.structured.Table; | ||
| import ru.fizteh.fivt.storage.structured.TableProvider; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.MultiFileHashMap.DatabaseFileStructureException; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.MultiFileHashMap.LoadOrSaveException; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.MultiFileHashMap.Pair; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.junit.DatabaseException; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.storeable.json.BrokenJsonException; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.storeable.json.JsonParser; | ||
| import ru.fizteh.fivt.students.LebedevAleksey.storeable.json.JsonUnsupportedObjectException; | ||
|
|
||
| import java.io.*; | ||
| import java.nio.file.Files; | ||
| import java.nio.file.Path; | ||
| import java.text.ParseException; | ||
| import java.util.*; | ||
| import java.util.concurrent.locks.Lock; | ||
| import java.util.concurrent.locks.ReadWriteLock; | ||
| import java.util.concurrent.locks.ReentrantReadWriteLock; | ||
| import java.util.function.BiConsumer; | ||
|
|
||
| public class Database implements TableProvider { | ||
| public static final String TABLE_SIGNATURE_FILE_NAME = "signature.tsv"; | ||
| private static final String INCORRECT_NAME_OF_TABLES = "This name is not correct, folder can't be created"; | ||
| private static Map<String, Class<?>> stringTypesMap = new HashMap<>(); | ||
| private static Map<Class, String> typesStringMap = new HashMap<>(); | ||
|
|
||
| static { | ||
| stringTypesMap.put("int", Integer.class); | ||
| stringTypesMap.put("long", Long.class); | ||
| stringTypesMap.put("byte", Byte.class); | ||
| stringTypesMap.put("float", Float.class); | ||
| stringTypesMap.put("double", Double.class); | ||
| stringTypesMap.put("boolean", Boolean.class); | ||
| stringTypesMap.put("String", String.class); | ||
| stringTypesMap.forEach(new BiConsumer<String, Class<?>>() { | ||
| @Override | ||
| public void accept(String alias, Class<?> type) { | ||
| typesStringMap.put(type, alias); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| private Path directoryPath; | ||
| private Map<String, StoreableTable> tables = new HashMap<>(); | ||
| private ReadWriteLock lock = new ReentrantReadWriteLock(); | ||
|
|
||
|
|
||
| public Database(String directory) throws IOException { | ||
| Lock writeLock = lock.writeLock(); | ||
| writeLock.lock(); | ||
| try { | ||
| assertArgumentNotNull(directory, "directory"); | ||
| File root = new File(directory); | ||
| directoryPath = root.toPath(); | ||
| File[] tables = root.listFiles(); | ||
| for (File file : tables) { | ||
| if (file.isDirectory()) { | ||
| File signature = file.toPath().resolve(TABLE_SIGNATURE_FILE_NAME).toFile(); | ||
| String tablename = file.getName(); | ||
| if (signature.exists() && signature.isFile()) { | ||
| String signatureString; | ||
| try (FileInputStream stream = new FileInputStream(signature.getAbsolutePath())) { | ||
| try (DataInputStream signaturedata = new DataInputStream(stream)) { | ||
| signatureString = signaturedata.readUTF(); | ||
| } | ||
| } | ||
| String[] tokens = signatureString.split(" "); | ||
| List<Class<?>> types = new ArrayList<>(); | ||
| if (signatureString.length() != 0) { | ||
| for (String item : tokens) { | ||
| Class<?> type = stringTypesMap.get(item); | ||
| if (type == null) { | ||
| throw new IOException("Wrong type name in signature of table " + tablename); | ||
| } else { | ||
| types.add(type); | ||
| } | ||
| } | ||
| } | ||
| StoreableTable table = generateTable(tablename, types); | ||
| this.tables.put(tablename, table); | ||
| } else { | ||
| throw new IOException("Where is not signature file in table " + tablename); | ||
| } | ||
| } else { | ||
| fileFoundInRootDirectory(file); | ||
| } | ||
| } | ||
| } finally { | ||
| writeLock.unlock(); | ||
| } | ||
| } | ||
|
|
||
| public static void throwIOException(Throwable e) throws IOException { | ||
| if (e.getCause() != null) { | ||
| try { | ||
| IOException ioException = (IOException) e.getCause(); | ||
| throw ioException; | ||
| } catch (ClassCastException wrongClass) { | ||
| //Not IOException | ||
| } | ||
| } | ||
| throw new IOException(e.getMessage(), e); | ||
| } | ||
|
|
||
| public static List<Class<?>> parseTypes(String[] input) throws ParseException { | ||
| List<Class<?>> types = new ArrayList<>(input.length); | ||
| for (String item : input) { | ||
| Class<?> type = stringTypesMap.get(item); | ||
| if (type != null) { | ||
| types.add(type); | ||
| } else { | ||
| throw new ParseException("Wrong type: " + item, -1); | ||
| } | ||
| } | ||
| return types; | ||
| } | ||
|
|
||
| protected void fileFoundInRootDirectory(File file) throws IOException { | ||
| throw new IOException("There is file " + file.getName() + " in root directory"); | ||
| } | ||
|
|
||
| @Override | ||
| public Table getTable(String name) { | ||
| assertArgumentNotNull(name, "name"); | ||
| Lock readLock = lock.readLock(); | ||
| readLock.lock(); | ||
| StoreableTable table = tables.get(name); | ||
| readLock.unlock(); | ||
| return table; | ||
| } | ||
|
|
||
| @Override | ||
| public Table createTable(String name, List<Class<?>> columnTypes) throws IOException { | ||
| assertArgumentNotNull(name, "name"); | ||
| assertArgumentNotNull(columnTypes, "columnTypes"); | ||
| Lock writeLock = lock.writeLock(); | ||
| writeLock.lock(); | ||
| try { | ||
| Table checkExists = getTable(name); | ||
| if (checkExists == null) { | ||
| Path rootDirectoryPath = getRootDirectoryPath(); | ||
| Path path = rootDirectoryPath.resolve(name); | ||
| if (path.startsWith(rootDirectoryPath) && path.getParent().equals(rootDirectoryPath)) { | ||
| if (columnTypes == null) { | ||
| throw new IllegalArgumentException("Argument \"columnTypes\" is null."); | ||
| } | ||
| String tableSignature = createTableSignature(columnTypes); | ||
| try { | ||
| Files.createDirectory(path); | ||
| try (FileOutputStream stream = new FileOutputStream(path.resolve( | ||
| TABLE_SIGNATURE_FILE_NAME).toString())) { | ||
| try (DataOutputStream dataStream = new DataOutputStream(stream)) { | ||
| dataStream.writeUTF(tableSignature); | ||
| dataStream.flush(); | ||
| } | ||
| } | ||
| } catch (Exception ex) { | ||
| try { | ||
| Files.delete(path); | ||
| } catch (Throwable suppressed) { | ||
| ex.addSuppressed(suppressed); | ||
| } | ||
| throw ex; | ||
| } | ||
| StoreableTable table = generateTable(name, columnTypes); | ||
| tables.put(name, table); | ||
| return table; | ||
| } else { | ||
| throw new IllegalArgumentException(INCORRECT_NAME_OF_TABLES); | ||
| } | ||
| } else { | ||
| return null; | ||
| } | ||
| } finally { | ||
| writeLock.unlock(); | ||
| } | ||
| } | ||
|
|
||
| private String createTableSignature(List<Class<?>> columnTypes) { | ||
| String tableSignature = ""; | ||
| for (Class<?> column : columnTypes) { | ||
| if (column == null) { | ||
| throw new IllegalArgumentException("Null column"); | ||
| } | ||
| String type = typesStringMap.get(column); | ||
| if (type == null) { | ||
| throw new IllegalArgumentException("Type is not supported"); | ||
| } | ||
| tableSignature += type; | ||
| tableSignature += " "; | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Используйте StringBuilder. Или += оптимизируется до StringBuilder? Что говорят перформансные тесты? |
||
| } | ||
| tableSignature = tableSignature.trim(); | ||
| return tableSignature; | ||
| } | ||
|
|
||
| public Path getRootDirectoryPath() { | ||
| return directoryPath; | ||
| } | ||
|
|
||
| private StoreableTable generateTable(String name, List<Class<?>> types) { | ||
| return new StoreableTable(name, this, types); | ||
| } | ||
|
|
||
| @Override | ||
| public void removeTable(String name) throws IOException { | ||
| assertArgumentNotNull(name, "name"); | ||
| Lock writeLock = lock.writeLock(); | ||
| writeLock.lock(); | ||
| try { | ||
| StoreableTable table = (StoreableTable) getTable(name); | ||
| if (table == null) { | ||
| throw new IllegalStateException("There is no table with name \"" + name + "\""); | ||
| } | ||
| try { | ||
| table.drop(); | ||
| } catch (DatabaseFileStructureException | LoadOrSaveException e) { | ||
| throwIOException(e); | ||
| } | ||
| tables.remove(name); | ||
| } finally { | ||
| writeLock.unlock(); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public Storeable deserialize(Table table, String value) throws ParseException { | ||
| assertArgumentNotNull(table, "table"); | ||
| assertArgumentNotNull(value, "value"); | ||
| List<Object> data; | ||
| try { | ||
| data = (List<Object>) JsonParser.parseJson(value); | ||
| } catch (BrokenJsonException e) { | ||
| throw new ParseException("Can't parse JSON: " + e.getMessage(), e.getOffsetError()); | ||
| } catch (ClassCastException e) { | ||
| throw new ParseException("Wrong JSON: not a list", 0); | ||
| } | ||
| if (data.size() == table.getColumnsCount()) { | ||
| Storeable storable = createFor(table); | ||
| for (int i = 0; i < data.size(); i++) { | ||
| if (data.get(i) == null) { | ||
| storable.setColumnAt(i, null); | ||
| } else if (data.get(i).getClass() == table.getColumnType(i)) { | ||
| storable.setColumnAt(i, data.get(i)); | ||
| } else if (!tryCastInteger(table, i, data.get(i), storable) | ||
| && !tryCastFloat(table, i, data.get(i), storable)) { | ||
| throw new ParseException("Wrong data type in column number " + i, -1); | ||
| } | ||
| } | ||
| return storable; | ||
| } else { | ||
| throw new ParseException("Wrong size of list: have " + data.size() + ", table have " | ||
| + table.getColumnsCount() + " columns.", value.length() - 1); | ||
| } | ||
| } | ||
|
|
||
| private boolean tryCastInteger(Table table, int column, Object value, Storeable result) { | ||
| if (table.getColumnType(column) == Integer.class || table.getColumnType(column) == Byte.class) { | ||
| if (value.getClass() == Long.class) { | ||
| if (table.getColumnType(column) == Integer.class) { | ||
| long val = (long) value; | ||
| if (val >= Integer.MIN_VALUE && val <= Integer.MAX_VALUE) { | ||
| result.setColumnAt(column, new Integer((int) val)); | ||
| return true; | ||
| } | ||
| } else { | ||
| if (table.getColumnType(column) == Byte.class) { | ||
| long val = (long) value; | ||
| if (val >= Byte.MIN_VALUE && val <= Byte.MAX_VALUE) { | ||
| result.setColumnAt(column, new Byte((byte) val)); | ||
| return true; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| private boolean tryCastFloat(Table table, int column, Object value, Storeable result) { | ||
| if (table.getColumnType(column) == Float.class) { | ||
| if (value.getClass() == Double.class) { | ||
| if (table.getColumnType(column) == Float.class) { | ||
| double val = (double) value; | ||
| if (Math.abs(val) >= Float.MIN_VALUE && val <= Float.MAX_VALUE) { | ||
| result.setColumnAt(column, (float) val); | ||
| return true; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| @Override | ||
| public String serialize(Table table, Storeable value) throws ColumnFormatException { | ||
| assertArgumentNotNull(table, "table"); | ||
| assertArgumentNotNull(value, "value"); | ||
| List<Object> data = new ArrayList<>(); | ||
| for (int i = 0; i < table.getColumnsCount(); i++) { | ||
| data.add(value.getColumnAt(i)); | ||
| } | ||
| try { | ||
| return JsonParser.getJson(data); | ||
| } catch (JsonUnsupportedObjectException e) { | ||
| throw new ColumnFormatException("Unknown column format", e); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public Storeable createFor(Table table) { | ||
| assertArgumentNotNull(table, "table"); | ||
| return new ru.fizteh.fivt.students.LebedevAleksey.storeable.Storeable(Arrays.asList( | ||
| new Object[table.getColumnsCount()]), table); | ||
| } | ||
|
|
||
| @Override | ||
| public Storeable createFor(Table table, List<?> values) throws ColumnFormatException, IndexOutOfBoundsException { | ||
| assertArgumentNotNull(table, "table"); | ||
| assertArgumentNotNull(values, "values"); | ||
| return new ru.fizteh.fivt.students.LebedevAleksey.storeable.Storeable((List<Object>) values, table); | ||
| } | ||
|
|
||
| @Override | ||
| public List<String> getTableNames() { | ||
| Lock readLock = lock.readLock(); | ||
| readLock.lock(); | ||
| final List<String> result = new ArrayList<>(tables.size()); | ||
| tables.keySet().forEach((String s) -> result.add(s)); | ||
| readLock.unlock(); | ||
| return result; | ||
| } | ||
|
|
||
| private void assertArgumentNotNull(Object argument, String name) { | ||
| if (argument == null) { | ||
| throw new IllegalArgumentException("Argument \"" + name + "\" is null"); | ||
| } | ||
| } | ||
|
|
||
| public List<Pair<String, Integer>> listTables() throws IOException { | ||
| final List<Pair<String, Integer>> result = new ArrayList<>(tables.size()); | ||
| Lock readLock = lock.readLock(); | ||
| readLock.lock(); | ||
| try { | ||
| tables.forEach(new BiConsumer<String, StoreableTable>() { | ||
| @Override | ||
| public void accept(String s, StoreableTable table) { | ||
| result.add(new Pair<String, Integer>(s, table.size())); | ||
| } | ||
| }); | ||
| } catch (DatabaseException e) { | ||
| throwIOException(e.getCause()); | ||
| } finally { | ||
| readLock.unlock(); | ||
| } | ||
| return result; | ||
| } | ||
| } | ||
16 changes: 16 additions & 0 deletions
16
src/ru/fizteh/fivt/students/LebedevAleksey/storeable/DatabaseFactory.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| package ru.fizteh.fivt.students.LebedevAleksey.storeable; | ||
|
|
||
| import ru.fizteh.fivt.storage.structured.TableProvider; | ||
| import ru.fizteh.fivt.storage.structured.TableProviderFactory; | ||
|
|
||
| import java.io.IOException; | ||
|
|
||
| /** | ||
| * Created by Алексей on 07.12.2014. | ||
| */ | ||
| public class DatabaseFactory implements TableProviderFactory { | ||
| @Override | ||
| public TableProvider create(String path) throws IOException { | ||
| return new Database(path); | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lock.readLock().lock();
Не нужно заводить переменную