kernel.impl.coreapijava
这个包包含核心API的直接实现。核心API是org.neo4j中定义的API。graphdb及其子包。 这里的类是实现细节,可能会在不通知的状况下更改。
IOnode
这是Neo4j IO抽象层。 它当前包含磁盘IO机制和文件分页。
因为历史缘由,这个模块包含了Neo4j的多个重要组件: - 嵌入的Java API - org.neo4j.graphdb - 嵌入的Java API实现 - org.neo4j.kernel.coreapi - org.neo4j.kernel.core - 嵌入的遍历Java API - org.neo4j.graphdb.traversal - 嵌入的遍历API实现 - org.neo4j.kernel.traversal - 批量导入 - org.neo4j.unsafe.impl.batchimport - 批处理插入器(遗留) - org.neo4j.unsafe.batchinsert - 事务状态构建层(“内核API”) - org.neo4j.kernel.api - org.neo4j.kernel.impl.api - 存储引擎 - org.neo4j.kernel.impl.store, - org.neo4j.kernel.impl.recovery - org.neo4j.kernel.impl.transaction - 配置 - org.neo4j.kernel.configuration - 常见的工具 - org.neo4j.helpers - org.neo4j.kernel.impl.util - org.neo4j.kernel.lifecycle - org.neo4j.kernel.monitoring - 锁 - org.neo4j.kernel.impl.locking - 内核扩展 - org.neo4j.kernel.extension
内核API管理与Neo4j内核的全部交互,主要由Cypher runtime 和 Core API。api
Usage缓存
这个接口的入口点是连接:src/main/java/org/neo4j/impl/kernel/api/Kernel.java[Kernel] 经过这里,咱们能够开始一个连接:src/main/java/org/neo4j/impl/kernel/api/Session.java[Session], 它容许咱们开始连接:src/main/java/org/neo4j/impl/kernel api/Transaction.java[Transactions]. 一旦进入事务,咱们能够直接访问内核的全部主要功能:服务器
使用游标读取app
读操做使用特殊的游标,这容许在热路径中很是低的对象分配率。ide
设计说明 一般,游标的设计方法是经过游标直接公开底层表示包含的全部信息。 其思想是将暴露给运行时的游标实现为直接位于页面数据之上的“视图”或“投影”,提供对存储的原始字节的解释。 不过,数据的公开并不会一直进行下去,而是试图在容许底层格式的将来更改之间取得平衡。 做为一个思想实验,我尝试以这样一种方式定义API,应该能够沿着这些方面改变实现:函数
在大多数状况下,经过使用不一样的游标类型显式地访问不一样的存储。 除了一个例外:link:src/main/java/ org/neo4j/impl/kernel/api/labelset。java[节点的标签]。工具
整体设计方法是让API的客户机控制游标的生命周期。 这比将游标集中在引擎内更有效。单元测试
这样作的缘由是,因为客户机是一个查询,因此使用模式是彻底已知的,所以能够显式地从池中取出和返回到池中。 一样,能够显式地在同一个查询中重用。可是不容许客户机提供本身的游标类型,必须使用连接提供的类型: link:src/main/java/org/neo4j/impl/kernel/api/CursorFactory.java[CursorFactory].
客户机控制游标的生命周期后,就排除了“Cursor”之类的通用接口,由于咱们须要游标是已知的具体类型。
“规范”用法是从其余游标初始化游标,而不是从记录中访问指针值,而后显式初始化游标。 指针值也能够访问,由于向量化的操做管道更喜欢这样的使用模式。
对底层表示的一个更改将使实现这个API更加容易,即便全部节点按照类型对它们的关系进行分组,而不只仅是密集节点。
此外,若是事务状态以与存储中相同的格式存储在堆外缓冲区中,则实现将更容易和更有效。
Neo4j UDC
Usage Data Collector (UDC)是一个内核扩展,它收集关于Neo4j服务器的统计信息。 详见: https://neo4j.com/docs/operations-manual/current/configuration/usage-data-collector/
procedure-compiler
Neo4j工具-过程|用户功能编译器 这是一个注释处理器,它将在编译时验证存储过程。 虽然能够执行大多数基本检查,可是仍然须要一些单元测试来验证一些运行时行为。 它是作什么的? 一旦将存储过程编译器添加到项目类路径中(请参阅下面的Maven/Gradle说明),若是不知足如下任何要求,它将触发编译失败: - `@Context` 字段必须是' public '且非' final ' - 全部其余字段必须是 `static` - `Map` 记录字段/过程参数必须将键类型定义为 `String` - `@Procedure`|`@UserFunction` 类必须定义一个没有参数的公共构造函数 - `@Procedure` 方法必须返回一个流Stream - `@Procedure`|`@UserFunction` 参数和记录类型必须支持 - `@Procedure`|`@UserFunction` 参数必须用 `@Name`标注 - `@UserFunction` 不能在根名称空间中定义 - 全部访问过的 `@Procedure`|`@UserFunction`名称必须是惟一的 *已部署的Neo4j实例能够聚合来自不一样jar的存储过程。 注释处理器没法检测到jar之间的命名冲突。 根据定义,它一次只能检查一个编译单元。
====== ## class StoreType
StoreType.java public enum StoreType { NODE_LABEL( DatabaseFile.NODE_LABEL_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createNodeLabelStore(); } }, NODE( DatabaseFile.NODE_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createNodeStore(); } }, PROPERTY_KEY_TOKEN_NAME( DatabaseFile.PROPERTY_KEY_TOKEN_NAMES_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyKeyTokenNamesStore(); } }, PROPERTY_KEY_TOKEN( DatabaseFile.PROPERTY_KEY_TOKEN_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyKeyTokenStore(); } }, PROPERTY_STRING( DatabaseFile.PROPERTY_STRING_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyStringStore(); } }, PROPERTY_ARRAY( DatabaseFile.PROPERTY_ARRAY_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyArrayStore(); } }, PROPERTY( DatabaseFile.PROPERTY_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyStore(); } }, RELATIONSHIP( DatabaseFile.RELATIONSHIP_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipStore(); } }, RELATIONSHIP_TYPE_TOKEN_NAME( DatabaseFile.RELATIONSHIP_TYPE_TOKEN_NAMES_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipTypeTokenNamesStore(); } }, RELATIONSHIP_TYPE_TOKEN( DatabaseFile.RELATIONSHIP_TYPE_TOKEN_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipTypeTokenStore(); } }, LABEL_TOKEN_NAME( DatabaseFile.LABEL_TOKEN_NAMES_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createLabelTokenNamesStore(); } }, LABEL_TOKEN( DatabaseFile.LABEL_TOKEN_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createLabelTokenStore(); } }, SCHEMA( DatabaseFile.SCHEMA_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createSchemaStore(); } }, RELATIONSHIP_GROUP( DatabaseFile.RELATIONSHIP_GROUP_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipGroupStore(); } }, COUNTS( DatabaseFile.COUNTS_STORES, false, false ) { @Override public CountsTracker open( NeoStores neoStores ) { return neoStores.createCountStore(); } @Override void close( Object object ) { try { ((CountsTracker) object).shutdown(); } catch ( IOException e ) { throw new UnderlyingStorageException( e ); } } }, META_DATA( DatabaseFile.METADATA_STORE, true, true ) // Make sure this META store is last { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createMetadataStore(); } };
ENUM_YPE | CALL_FUNC |
---|---|
NODE_LABEL | createNodeLabelStore |
NODE | createNodeStore |
PROPERTY_KEY_TOKEN_NAME | createPropertyKeyTokenNamesStore |
… | … |
该类包含对“NodeStore、RelationshipStore、PropertyStore和RelationshipTypeStore”的引用。 实际上,NeoStores并不“存储”任何东西,而是为在其中执行的“类型和版本”验证扩展了AbstractStore。
CommonAbstractStore createNodeStore() { return initialize( new NodeStore( layout.nodeStore(), layout.idNodeStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicArrayStore) getOrCreateStore( StoreType.NODE_LABEL ), recordFormats, openOptions ) ); } CommonAbstractStore createNodeLabelStore() { return createDynamicArrayStore( layout.nodeLabelStore(), layout.idNodeLabelStore(), IdType.NODE_LABELS, GraphDatabaseSettings.label_block_size ); } CommonAbstractStore createPropertyKeyTokenStore() { return initialize( new PropertyKeyTokenStore( layout.propertyKeyTokenStore(), layout.idPropertyKeyTokenStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.PROPERTY_KEY_TOKEN_NAME ), recordFormats, openOptions ) ); } CommonAbstractStore createPropertyKeyTokenNamesStore() { return createDynamicStringStore( layout.propertyKeyTokenNamesStore(), layout.idPropertyKeyTokenNamesStore(), IdType.PROPERTY_KEY_TOKEN_NAME, TokenStore.NAME_STORE_BLOCK_SIZE ); } CommonAbstractStore createPropertyStore() { return initialize( new PropertyStore( layout.propertyStore(), layout.idPropertyStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.PROPERTY_STRING ), (PropertyKeyTokenStore) getOrCreateStore( StoreType.PROPERTY_KEY_TOKEN ), (DynamicArrayStore) getOrCreateStore( StoreType.PROPERTY_ARRAY ), recordFormats, openOptions ) ); } CommonAbstractStore createPropertyStringStore() { return createDynamicStringStore( layout.propertyStringStore(), layout.idPropertyStringStore(), IdType.STRING_BLOCK, GraphDatabaseSettings.string_block_size ); } CommonAbstractStore createPropertyArrayStore() { return createDynamicArrayStore( layout.propertyArrayStore(), layout.idPropertyArrayStore(), IdType.ARRAY_BLOCK, GraphDatabaseSettings.array_block_size ); } CommonAbstractStore createRelationshipStore() { return initialize( new RelationshipStore( layout.relationshipStore(), layout.idRelationshipStore(), config, idGeneratorFactory, pageCache, logProvider, recordFormats, openOptions ) ); } CommonAbstractStore createRelationshipTypeTokenStore() { return initialize( new RelationshipTypeTokenStore( layout.relationshipTypeTokenStore(), layout.idRelationshipTypeTokenStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.RELATIONSHIP_TYPE_TOKEN_NAME ), recordFormats, openOptions ) ); } CommonAbstractStore createRelationshipTypeTokenNamesStore() { return createDynamicStringStore( layout.relationshipTypeTokenNamesStore(), layout.idRelationshipTypeTokenNamesStore(), IdType.RELATIONSHIP_TYPE_TOKEN_NAME, TokenStore.NAME_STORE_BLOCK_SIZE ); } CommonAbstractStore createLabelTokenStore() { return initialize( new LabelTokenStore( layout.labelTokenStore(), layout.idLabelTokenStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.LABEL_TOKEN_NAME ), recordFormats, openOptions ) ); } CommonAbstractStore createSchemaStore() { return initialize( new SchemaStore( layout.schemaStore(), layout.idSchemaStore(), config, IdType.SCHEMA, idGeneratorFactory, pageCache, logProvider, recordFormats, openOptions ) ); } CommonAbstractStore createRelationshipGroupStore() { return initialize( new RelationshipGroupStore( layout.relationshipGroupStore(), layout.idRelationshipGroupStore(), config, idGeneratorFactory, pageCache, logProvider, recordFormats, openOptions ) ); } CommonAbstractStore createLabelTokenNamesStore() { return createDynamicStringStore( layout.labelTokenNamesStore(), layout.idLabelTokenNamesStore(), IdType.LABEL_TOKEN_NAME, TokenStore.NAME_STORE_BLOCK_SIZE ); } CountsTracker createCountStore() { boolean readOnly = config.get( GraphDatabaseSettings.read_only ); CountsTracker counts = readOnly ? createReadOnlyCountsTracker( layout ) : createWritableCountsTracker( layout ); NeoStores neoStores = this; counts.setInitializer( new DataInitializer<CountsAccessor.Updater>() { private final Log log = logProvider.getLog( MetaDataStore.class ); @Override public void initialize( CountsAccessor.Updater updater ) { log.warn( "Missing counts store, rebuilding it." ); new CountsComputer( neoStores, pageCache, layout ).initialize( updater ); log.warn( "Counts store rebuild completed." ); } @Override public long initialVersion() { return ((MetaDataStore) getOrCreateStore( StoreType.META_DATA )).getLastCommittedTransactionId(); } } ); try { counts.init(); // TODO: move this to LifeCycle } catch ( IOException e ) { throw new UnderlyingStorageException( "Failed to initialize counts store", e ); } return counts; } CommonAbstractStore createMetadataStore() { return initialize( new MetaDataStore( metadataStore, layout.idMetadataStore(), config, idGeneratorFactory, pageCache, logProvider, recordFormats.metaData(), recordFormats.storeVersion(), openOptions ) ); } private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicStringStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, int blockSize ) { return initialize( new DynamicStringStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats.dynamic(), recordFormats.storeVersion(), openOptions ) ); } private CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicArrayStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, int blockSize ) { if ( blockSize <= 0 ) { throw new IllegalArgumentException( "Block size of dynamic array store should be positive integer." ); } return initialize( new DynamicArrayStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats, openOptions ) ); }
createCountStore() | createCountStore() |
… | CommonAbstractStore |
显然,仅有createCountStore() 返回 createCountStore().其他均为CommonAbstractStore.
createNodeStore | initialize(new NodeStore()) |
createPropertyKeyTokenStore | initialize( new PropertyKeyTokenStore()) |
createPropertyStore | initialize( new PropertyStore()) |
createRelationshipStore | initialize(new RelationshipStore()) |
createRelationshipTypeTokenStore | initialize(new RelationshipTypeTokenStore()) |
createLabelTokenStore | initialize(new LabelTokenStore()) |
createSchemaStore | initialize(new SchemaStore()) |
createRelationshipGroupStore | initialize( new RelationshipGroupStore()) |
createMetadataStore | initialize(new MetaDataStore()) |
createDynamicStringStore | initialize( new DynamicStringStore()) |
createDynamicArrayStore | initialize( new DynamicArrayStore()) private 检查blockSize |
createNodeLabelStore | createDynamicArrayStore() |
createPropertyArrayStore | createDynamicArrayStore() |
createDynamicArrayStore | createDynamicArrayStore() |
createPropertyKeyTokenNamesStore | createDynamicStringStore() |
createPropertyStringStore | createDynamicStringStore() |
createRelationshipTypeTokenNamesStore | createDynamicStringStore() |
createLabelTokenNamesStore | createDynamicStringStore() |
createDynamicStringStore | createDynamicStringStore() |
createCountStore | special |
NeoStores.java private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicStringStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, int blockSize ) { return initialize( new DynamicStringStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats.dynamic(), recordFormats.storeVersion(), openOptions ) ); } private CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicArrayStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, int blockSize ) { if ( blockSize <= 0 ) { throw new IllegalArgumentException( "Block size of dynamic array store should be positive integer." ); } return initialize( new DynamicArrayStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats, openOptions ) ); }
可见,这里都是直接或间接调用 initialize(new <type>(args…)) 实现的.
NeoStores.java private <T extends CommonAbstractStore> T initialize( T store ) { store.initialise( createIfNotExist ); return store; }
CommonAbstractStore.java void initialise( boolean createIfNotExists ) { try { checkAndLoadStorage( createIfNotExists ); } catch ( Exception e ) { closeAndThrow( e ); } }
NeoStores类包含对“NodeStore、RelationshipStore、PropertyStore和RelationshipTypeStore”的引用。 实际上,NeoStores并不“存储”任何东西,而是为在其中执行的“类型和版本”验证扩展了AbstractStore。
CommonAbstractStore.java /** * This method is called by constructors. Checks the header record and loads the store. * <p> * Note: This method will map the file with the page cache. The store file must not * be accessed directly until it has been unmapped - the store file must only be * accessed through the page cache. * @param createIfNotExists If true, creates and initialises the store file if it does not exist already. If false, * this method will instead throw an exception in that situation. */ protected void checkAndLoadStorage( boolean createIfNotExists ) { int pageSize = pageCache.pageSize(); int filePageSize; try ( PagedFile pagedFile = pageCache.map( storageFile, pageSize, ANY_PAGE_SIZE ) ) { extractHeaderRecord( pagedFile ); filePageSize = pageCache.pageSize() - pageCache.pageSize() % getRecordSize(); } catch ( NoSuchFileException | StoreNotFoundException e ) { if ( createIfNotExists ) { try { createStore( pageSize ); return; } catch ( IOException e1 ) { e.addSuppressed( e1 ); } } if ( e instanceof StoreNotFoundException ) { throw (StoreNotFoundException) e; } throw new StoreNotFoundException( "Store file not found: " + storageFile, e ); } catch ( IOException e ) { throw new UnderlyingStorageException( "Unable to open store file: " + storageFile, e ); } loadStorage( filePageSize ); }
此方法由构造函数调用。检查头记录并加载存储。
注意:这个方法将用页面缓存映射文件。在未映射以前,不能直接访问存储文件——只能经过页面缓存访问存储文件。 @param createIfNotExists(若是为真)建立并初始化存储文件(若是它还不存在)。若是为false,则此方法将在这种状况下抛出异常。 这里调用createStore(pageSize)建立存储文件.
CommonAbstractStore.java private void createStore( int pageSize ) throws IOException { try ( PagedFile file = pageCache.map( storageFile, pageSize, StandardOpenOption.CREATE ) ) { initialiseNewStoreFile( file ); } checkAndLoadStorage( false ); } protected void initialiseNewStoreFile( PagedFile file ) throws IOException { if ( getNumberOfReservedLowIds() > 0 ) { try ( PageCursor pageCursor = file.io( 0, PF_SHARED_WRITE_LOCK ) ) { if ( pageCursor.next() ) { pageCursor.setOffset( 0 ); createHeaderRecord( pageCursor ); if ( pageCursor.checkAndClearBoundsFlag() ) { throw new UnderlyingStorageException( "Out of page bounds when writing header; page size too small: " + pageCache.pageSize() + " bytes." ); } } } } // Determine record size right after writing the header since some stores // use it when initializing their stores to write some records. recordSize = determineRecordSize(); idGeneratorFactory.create( idFile, getNumberOfReservedLowIds(), false ); } private void createHeaderRecord( PageCursor cursor ) { int offset = cursor.getOffset(); storeHeaderFormat.writeHeader( cursor ); cursor.setOffset( offset ); readHeaderAndInitializeRecordFormat( cursor ); } /** * This method is called when opening the store to extract header data and determine things like * record size of the specific record format for this store. Some formats rely on information * in the store header, that's why it happens at this stage. * * @param cursor {@link PageCursor} initialized at the start of the store header where header information * can be read if need be. This can be {@code null} if this store has no store header. The initialization * of the record format still happens in here. * @throws IOException if there were problems reading header information. */ private void readHeaderAndInitializeRecordFormat( PageCursor cursor ) { storeHeader = storeHeaderFormat.readHeader( cursor ); }
IntStoreHeaderFormat.java @Override public IntStoreHeader readHeader( PageCursor cursor ) { return new IntStoreHeader( cursor.getInt() ); }
[ ] 发现了 readme.md 或许有点用