23. Release Notes

23.1. Experimental features

From time to time, RDFox® releases may include experimental features not intended for use in production. In these cases, the affected APIs will clearly be labeled “EXPERIMENTAL” meaning that they may change or disappear entirely without notice in later versions. Further, Oxford Semantic Technologies offers no support for any API labeled in this way.

If you have feedback or questions regarding any feature of RDFox, whether experimental or not, we would be grateful to hear it via any of the normal channels.

23.2. Releases

Release 7.2b (28th November 2024)

  • Fixed a crash when reporting errors from a multi-threaded import operation (RD-2069). This issue can only occur when the target data store has its invalid-literal-policy property set to as-string or as-string-silent.

  • Fixed a crash when evaluating SPARQL updates with attached query monitor (RD-2049).

  • Fixed an issue with presenting incorrect information about rules and axioms after loading a data store from a binary format (RD-2050, RD-2034).

  • Fixed an issue with timezone conversions (TO_TIMEZONE, fn:adjust-dateTime-to-timezone, fn:adjust-date-to-timezone, and fn:adjust-time-to-timezone) where the resulting value could incorrectly account for the existing timezone of the provided time/datetime (RD-2055).

  • Fixed an issue with the persistence of resource specifiers ending |namedgraphs|* (RD-2059). The fault led to these specifiers being persisted as if they ended |namedgraphs|UNDEF. After a restart, this would result in the role to which the privilege was granted, and all of its member roles, being unable to run any SPARQL query involving named graphs. See the instructions associated with the release for details of how to recover servers affected by this issue.

  • Ensured delta queries cannot be registered with an invalid answer format. (RD-1957).

Release 7.2a (28th October 2024)

  • Fix a crash when multiplying large XSDDecimal (RD-1997)

  • Fix a correctness issue when evaluating filter expressions nested in (NOT) EXISTS (RD-1998)

  • Delta queries:

    • Fix an issue in which delta query answers were not deleted on query deregistration (RD-1952)

    • Added the ability for delta queries to take empty snapshots (RD-1955)

    • Fix a crash when registering certain delta queries with DISTINCT (RD-2017)

    • Fix an issue with loading data stores from binary formats when they have registered delta queries (RD-2027)

Release 7.2 (22nd August 2024)

  • The RDFox Console has been redesigned to improve usability (RD-1755).

  • Added an EXPERIMENTAL feature to track changes to a knowledge graph known as ‘delta queries’ (RD-1744). See Section 11.7.

  • Added support for upgrading persistent servers to the persistence format of the current RDFox version (RD-1868). See Section 13.1.2. The upgrade feature is currently unable to upgrade data stores that have had a data source registered since the most recent snapshot. This can be worked around by deregistering all data sources and then compacting in the lower RDFox version, before attempting the upgrade.

  • Compaction now automatically recovers disk space in all cases (RD-1844). The delete-redundant-files parameter of the operation has been removed.

  • Roles can now be created without passwords (RD-1815). See Section 16.18 for details of the APIs for creating roles and Section 15.2.38 for how to create roles in the RDFox shell.

  • Improved the automatic management of database connections in PostgreSQL and ODBC data sources (RD-1873).

  • Attempting to bring a data store offline now returns an enum, indicating whether the operation was successful, the data store was already offline, or that the data store was in use and could not be brought offline (RD-1773).

  • The server directories of persistent servers must now always be initialized before a server can be started. In versions 7.0 and 7.1, explicit initialization was only required for servers managed using the RDFox executable but now servers managed via RDFox’s Java API must also be initialized. As part of this change, the ConnectionFactory class from the tech.oxfordsemantic.jrdfox.client package has been replaced by the new RDFoxServer class which exposes methods to support analyzing, initializing, starting, and stopping RDFox servers. See Section 16.1.

  • The Java API method getVersion() has been replaced with getRDFoxVersion(), and the method getGitSHA() has been replaced with getRDFoxGitSHA(). This change was made to avoid confusion with the newly introduced getServerVersion() method in the ServerConnection .

  • The following server parameters renames have been applied:

    • file-system-poll-interval -> persistence.file-system-poll-interval

    • license-content -> license.content

    • license-file -> license.file

    • notifications-address -> persistence.notifications-address

    • persistence.encryption-algorithm -> persistence.encryption.algorithm

    • persistence.encryption-key -> persistence.encryption.key

  • The role-manager option for the authentication-managers server parameter has been renamed to role-name+password.

  • The lowest supported version of macOS has been increased from 10.14 to 10.15.

  • The base layer for the official Docker images has been updated from Ubuntu 22.04 to Ubuntu 24.04.

  • The mechanism for loading console extensions, such as faceted search has been retired (RD-1755).

  • Fixed a flaw in access control where properties of a resource that would not be accessible to an agent trying to read the resource directly were returned to the agent when listing all resources of that type (RD-1799). This affected data stores, data sources, and tuple tables. List operations for these resources will now return the names of all items, but additional properties will only be returned for those individuals where the agent holds read privileges.

  • Fixed an issue with parameter handling in the importaxioms shell command (RD-1801).

  • Fixed an issue with the import, importaxioms, and info shell commands where the provided named graph was not being resolved against the base IRI (RD-1790).

  • Fixed an issue with the reasoning profiler where rule deletion may lead to a crash (RD-1782).

  • Fixes to the executable’s init mode to ensure that:

    • the process either succeeds or leaves the filesystem unchanged (RD-1792)

    • errors returned by any included shell commands cause the process to return a non-zero exit code (RD-1802).

  • Addressed issues in the reasoning profiler:

    • Fixed a bug where printing intermediate logs could cause a crash (RD-1812).

    • Corrected the miss-ordering of plans (RD-1852).

  • Fixed a bug where every role was counted as being a member of itself, allowing the roles registered as externally authenticatable or grantable to be passed directly by the identity provider (RD-1921).

Release 7.1b (26th July 2024)

  • Eliminated all security vulnerabilities currently known to AWS ECR’s scanner from the Docker images by rebuilding with the latest Ubuntu 22.04 base image.

  • Fixed an issue where restoring a change in which a rule was deleted, on an instance persisted with the file-sequence persistence type, could cause the in-memory data store to become corrupted (RD-1869).

  • Fixed a crash that could occur when an explanation was requested for a malformed fact (RD-1886).

  • Added calls to malloc_trim after compaction and data store deletion (Linux only). This is to ensure that memory freed by the RDFox program code is returned to the OS. We believe this is only needed because of a bug in glibc (reported here).

Release 7.1a (10th May 2024)

  • Added server parameters specifying the time, memory, and parallelism costs of Argon2i password hashing (RD-1797). Setting all three during server initialization bypasses RDFox’s automatic determination of these parameters. See Section 12.2.1.1.3.

  • Fixed a bug in compaction where resources that were only mentioned in rules were lost from the dictionary, leading those rules to potentially derive invalid facts in the place of correct facts (RD-1794).

  • Fixed a bug where shutting down the RDFox server explicitly using the Java or C APIs while one or more data stores were offline would result in a crash (RD-1785).

Release 7.1 (11th April 2024)

  • Added support for OpenID Connect authentication (RD-1713). See Section 12.2.1.2.

  • Added new JRDFox API methods to support import and export of data in an encrypted format (RD-1746). New cipher streams were also added to the tech.oxfordsemantic.jrdfox.client.io package, to support reading and writing of this encrypted data.

  • A new icon is displayed in the Console’s data store selector with the names of data stores that were created with certain data store parameters (Section 5.4.1). The icon appears if one or more of the parameters that affect the semantics of the data store, namely equality, swrl-negation-as-failure and default-graph-name, is set to a value other than its default value. A tool tip on the new icon displays these customizations.

  • Improved the evaluation of queries and rules that refer to named graphs (RD-1756).

  • The icons next to individual IRIs and blank nodes in the Console’s query results table now only appear if that resource is the subject or object of a triple in the current data store. In previous versions the icon would appear even if the resource was only present as a predicate and therefore did not correspond to a starting point for the Explore mode (RD-1728).

  • Optimized the operation to recompute materialization (as invoked by the remat command) to avoiding triggering an unnecessary incremental reasoning phase before the full rematerialization.

  • Optimized the clearing of rules and axioms so that, when the clear operation leaves the rule index empty, no reasoning is run.

  • Changed the explanation format in the REST API and fixed an issue with the explain visualization of ground rules that use aggregation or negation.

  • Creating a data source tuple table now requires a read privilege over the data source (RD-1749).

  • The tuple table creation parameter dataSourceName has been deprecated and replaced with data-source-name (RD-1764). See Section 7.2 and sub-sections thereof.

  • The proprietary built-in ROLE function has been renamed to AGENT (RD-1713).

  • The proprietary x-role-name field in the Extended Log Format (ELF) has been renamed to x-agent-name (RD-1713).

  • The check password operation on connections, which was exposed in REST and Java, has been removed (RD-1713).

  • The tuple table creation parameter dataSourceName has been deprecated and replaced with data-source-name (RD-1764). See Section 7.2 and sub-sections thereof.

  • The Java API methods getRoleName() (present on both server and data store connections) has been replaced with getAgentName() (RD-1713).

  • Fixed a bug where TriG exports from data stores with the default-graph-name parameter contained plain triples duplicating those in all named graphs (RD-1727).

Release 7.0a (31st January 2024)

  • Fixed a crash after a rollback of a read-write transaction that adds new resources (RD-1722).

  • Open the correct type of transaction in the explain shell command (RD-1721).

Release 7.0 (9th January 2024)

  • RDFox now features improved concurrency management, where a writer never blocks readers (RD-1451).

  • RDFox servers now support encryption of all persisted data (RD-1641). See the persistence.encryption-key and persistence.encryption-algorithm server parameters in Section 4.3.

  • The RDFox endpoint and remote shell client now support authentication using client certificates (RD-1635). See the new request-client-certificates and trusted-root-certificate parameters to the endpoint in Section 19.2 and the available parameters for the remote shell client in Section 18.7.

  • Data stores with persistence enabled can now be brought offline in order to free up memory, and brought back online when they are needed again (RD-1650). See Section 15.2.16 for details of how to do this in the shell and Section 16.4.5 for how to do it programmatically. The newly-introduced server parameter all-data-stores-online-on-startup can be set to false to start the server with all data stores initially offline.

  • RDFox now supports installation as a background service via the new service executable mode (RD-1662). In support of this change a second new executable mode, init has been added to handle initialization of server directories, including writing configuration files and initializing persistence. See Section 18 for instructions on how to use all modes of the RDFox executable.

  • The RDFox Update Protocol now supports selective replacement of data store content (e.g., replacing just the rules in one RESTful API call); see Section 16.9.2.6.

  • The /content key now implements the SPARQL 1.1 Graph Store HTTP Protocol fully, as well as a proprietary RDFox Update Protocol (see Section 16.9).

  • RDFox now supports automatic compaction and automatic dead fact removal to help reclaim memory and minimise restart times after facts are deleted from a data store (see Section 5.2.4).

  • Introduced the notion of data store properties that represent various aspects of the configuration of a data store. Many configuration options (e.g., import.invalid-literal-policy) that were immutable in earlier versions are now exposed as mutable data store properties (see Section 5.4.2).

  • Assorted improvements to the RDFox console SPARQL editor (RD-1577).

  • Added sampling information to the query and reasoning profilers (RD-1617). See Section 9.6 and Section 10.9 for instructions on how to use these tools.

  • Added an endpoint configuration parameter lock-timeout that controls how long the endpoint will wait to obtain a lock on a data store (RD-1685). See the full description in Section 19.2.

  • The default-graph-name data store parameter is no longer EXPERIMENTAL. The console has been updated to allow the parameter to be set and to work with data stores that have a value for the parameter (RD-1595).

  • When using the RESTful API, data store operations such as clearing, compacting, etc., are now invoked by sending POST requests to path /datastores/<DSTRNAME>/operations rather than to /datastore/<DSTRNAME>. See Section 16.4.8 for the complete list of operations available using the new path (RD-1640).

  • The server parameters persist-roles and persist-ds, which formerly controlled persistence of roles and data stores respectively, have been replaced by a single parameter persistence which controls persistence of all server content. The data store parameter persist-ds has been renamed to persistence to match.

  • The server parameters file name has changed from parameters to server.params and an equivalent file for endpoint parameters has been introduced with name endpoint.params.

  • CORS support in the RDFox endpoint has been improved to allow authenticated cross-origin requests for all origins or a list of specific origins (RD-1687). See the access-control-allow-origin endpoint parameter in Section 19.2.

  • Added extensions to the aggregate functions MIN and MAX (RD-1680). See the full list of aggregate functions in Section 9.3.

  • Added support for version 3 of OpenSSL libraries and discontinued support for lower versions (RD-1637).

  • Parameters for setting the search paths for specific dynamic libraries (specifically the endpoint parameters openSSL.crypto-lib and openSSL.ssl-lib, and the driver and driver-manager data source parameters) have been replaced with equivalent environment variables. See Section 2.4.1.2.

  • Deprecated the base command in favor of dsprop.

  • Removed the daemon shell command. The daemon executable mode is still supported.

  • The clear command, when run without arguments, now clears all parts of the data store (SUP-186).

  • The proprietary Extended Log Format (ELF) property name x-socket-exception has been replaced by x-protocol-error.

  • Removed optimised handling of equality (i.e., owl:sameAs) and replaced it by an explicit axiomatisation using rules.

  • Removed the chase option for the equality data store parameter.

  • Renamed the shell variables reason.profiler-entries and reason.profiler-logPlans to reason.profiler.entries and reason.profiler.log-plans, respectively.

  • rdfox-init Docker images no longer copy a parameters file into the server directory. Instead, all server and endpoint parameters which should be saved to the server directory being initialized should be specified in the command arguments (RD-1668).

  • The rdfox Docker images no longer set the request-logger endpoint parameter by default (RD-1668).

  • Fixed a bug in the evaluation of SPARQL queries with graph clauses (RD-1657).

Release 6.3b (24th October 2023)

  • Fixed a crash when using Solr data sources, which was introduced in v6.3 (RD-1618).

    • Setting the Solr data source parameter connection-keep-alive to 0 now disables the caching of connections to the Apache Solr server.

  • Fix a bug that introduced spurious triples when rules containing atoms with repeated variables were added incrementally on a server with more than one thread (SUP-180).

  • Fixed a crash in compaction on stores with rules imported by importaxtioms (SUP-182).

  • Other small fixes and optimizations in the engine.

Release 6.3a (9th August 2023)

  • Fixed a crash when explaining facts derived by rules containing SKOLEM atoms (RD-1589).

  • Fixed a bug which caused incorrect answers to be returned for some queries with FILTER EXISTS or FILTER NOT EXISTS clauses (SUP-176).

  • Fixed a bug in the equality-optimization planning algorithm which caused incorrect answers to be returned for some queries that bind a variable to the value of another variable (RD-1588).

  • Fixed a bug in argument handling for shell scripts where invoking one script from another could incorrectly delete the positional arguments from the context of the calling script (RD-1599).

  • Other fixes in the engine and console.

Release 6.3 (26th July 2023)

  • Added an “Explain” mode to the RDFox console to graphically depict and explore the explanation of a derived fact. This may be accessed from the mode selector at the top right of the Console, or from a context menu on an edge representing a derived fact in the Explore mode. See Section 10.8.2.

  • Added support for listing, inspecting and cancelling RESTful requests running on the RDFox endpoint.

  • The compact operation now compacts the dictionary to recover memory and resource ID space.

  • Added an EXPERIMENTAL facility to specify a SPARQL update to be run on the closure of each data store transaction. See Section 11.6.

  • Added an EXPERIMENTAL data store parameter to specify a named graph to act as the default graph. See Section 5.4.1.1.

  • Removed the sequential data store type. Since all remaining data store types now support concurrent access, the JRDFox API tech.oxfordsemantic.jrdfox.client.DataStoreConnection.isConcurrent has also been removed.

  • Renamed all APIs and commands for triggering recompilation of a data store’s rules from recompileRules (or similar) to recompile and extended the implementation to also recompile any commit procedure.

  • Changed all APIs and commands for setting a data store’s base IRI to return a boolean indicating whether the value changed or not.

  • SPARQL FROM statements now compute the multiset union of triples in the selected named graphs as was the case prior to v6.0.

  • Fixed bug where compacting a file-sequence data store could lead to persistence of transient dictionary resources, preventing the sequence from being loaded by other replicas (RD-1546).

  • Fixed a bug where new versions of data stores and role managers were not loaded after a commit failed due to lagging replication (RD-1499).

  • Fixed the datalog parser in JRDFox to accept aggregate rules without ON clauses (RD-1511).

  • Fixed handling of BASE directives in Turtle and TriG inputs (RD-1489).

  • Fixed a bug with serialization of axioms in deltas. The bug would be encountered in transactions were axioms were added to and removed from the same named graph (SUP-175).

  • Fixed the TLS v1.3 client implementation to include the Server Name Indication (SNI) extension in the handshake (RD-1528).

  • Fixed a crash in query answering affecting queries with EXISTS under BIND (RDF-1554).

  • Fixed the rejection of certain queries that use EXISTS (RDF-1554).

Release 6.2 (29th March 2023)

  • The remote mode of the RDFox executable now supports cookies so that it can be used with high availability RDFox services where the load balancer supports sticky sessions (RD-1459).

  • Fixed an issue where the same sequence of blank node identifiers was repeated in each persistence session, potentially leading to properties being added to existing rather than new nodes (RD-1485).

  • Fixed an issue where the values for the Date header in HTTP responses from the RDFox endpoint were local times rather than GMT (RD-1486).

Release 6.1 (22nd March 2023)

  • The RDFox Console now supports large numbers of named graphs through the following changes:

    • “Add content” and “Delete content” menus support named graph searching.

    • Named graphs can be created as part of the “Add content” action.

    • If there are more named graphs in the datastore than can be listed, the “Explore” mode named graph selector now provides a search interface.

  • Added access control for named graphs. See Section 12.3.2.2.

  • All APIs (Java, C and REST) that returned or resolved dictionary resource IDs have been removed, as has the lookup shell command.

  • Resource names of the form |datastores|{data-store-name}|dictionary are no longer supported.

  • The import.invalid-literal-policy was renamed to invalid-literal-policy and is now applicable to queries and updates. See Section 5.4.2.5.

  • Added variants of the SHACL and DependencyGraph tuple tables that provide access to the default graph. See Section 6.5.2 and Section 6.5.3 respectively.

  • The clear operation (invoked, for example, by the clear shell command) no longer clears the data store’s prefixes or resets its base IRI. The same operation did take these actions in RDFox v6.0 (only) however the online help and published documentation did not mention this behavior (SUP-157).

  • RDFox servers now perform the initial restoration of content persisted with the file-sequence persistence option as part of the start operation rather than in the background. This is consistent with the behavior when using the file persistence option. For the RDFox executable, the endpoint (daemon mode) or shell (shell and sandbox modes) are started after the server start operation has completed.

  • Optimized the evaluation of queries that bind variables to RDF terms or other variables.

  • Fixed validation of endpoint parameters to allow openSSL.crypto-lib and openSSL.ssl-lib parameters to be set (RD-1423).

  • Fixed the ResourceValue.getResource(...).toResource() method in JRDFox to return objects of type IRI and blank node when returning IRIs and blank nodes

  • Fixed an issue with evaluation of queries that have unbound variables in their selection (RD-1436).

  • Fixed return code from oxfordsemantic/rdfox-init images when an error stops execution of an initialize.rdfox script (RD-1472).

  • Fixed the escaping of xsd:AnyURI values in N-Triples and N-Quads query answer and export formats.

  • Fixed a crash in file-sequence persistence when UDP notifications are configured (RD-1479).

  • The following changes were made to the class tech.oxfordsemantic.jrdfox.client.StatementResult:

    • The methods getNumberOfTuples() and getTotalNumberOfTuples() have been replaced with getNumberOfQueryAnswers() and getTotalNumberOfQueryAnswers() respectively.

    • The following new public methods have been added:

      • boolean getStatementTypeSupportsUserQuery()

      • boolean getStatementTypeSupportsDeletions()

      • long getNumberOfAttemptedDeletions()

      • long getNumberOfDeletions()

      • boolean getStatementTypeSupportsInsertions()

      • long getNumberOfAttemptedInsertions()

      • long getNumberOfInsertions()

Release 6.0 (23rd November 2022)

  • A new persistence type, file-sequence, has been added to facilitate high-availability deployments. See Section 13.2.1.2.

  • Improved the handling of large numbers of named graphs.

    • Named graphs are no longer stored in individual tuple tables of type named-graph, but are instead stored in a single tuple table called Quads. See Section 6.4. This reduces the memory overheads per named graph and removes the practical limits on the number of named graphs in a data store.

    • Graph names are no longer restricted to IRIs. RDFox now accepts arbitrary RDF terms as graph names in RDF data, SPARQL queries, datalog rules and relevant REST/Java/C APIs.

    • The shell commands info axioms, import and importaxioms, accept graph names specified either as IRIs or as strings containing the Turtle encoding of RDF terms.

    • The REST parameters default-graph, source-graph and target-graph replace the old parameters default-graph-name, source-graph-name and target-graph-name, respectively. When specified, they should contain the Turtle encoding of an RDF term.

    • The new data store parameter quad-table-type determines the type of the Quads table.

    • The rule language has been extended with named graph atoms for referring to triples in named graphs. See Section 10.4.1.2.

  • Each data store is now associated with a base IRI and a set of prefixes that are assumed to be implicitly present in all API calls. All RDFox APIs have been extended to support management of this information (see Section 16.6). New data stores are populated with a set of default prefixes and the default base IRI https://rdfox.com/default-base-iri/. RDFox shell instances no longer maintain a base IRI nor a set of prefixes and the base, prefix and prefixes shell commands have been repurposed to operate on the corresponding properties of the data store instead. Finally, the RDFox Console has been extended with interfaces to display and modify data store prefixes and to display IRIs in their prefixed form wherever this makes sense.

  • The executable and dynamic-link library for macOS and Windows are now code-signed. For macOS, these items are also notarized by Apple.

  • The REST APIs and RDFox executable mode related to remote shell execution, which were introduced as EXPERIMENTAL features in v5.6, are now supported.

  • The following types of in-memory tuple tables have been added to RDFox: unary-table, binary-table, triple-table, quad-table-lg, and quad-table-sg. The tuple table type named-graph has been removed.

  • Data store types have been changed to sequential, parallel-nn, parallel-nw and parallel-ww. Data store type seq has been replaced with sequential, while data store types par-complex-* and par-simple-* have been replaced with parallel-*.

  • Tuple table names are no longer IRIs but strings, similar to data store names and data source names. The following tuple tables are available after data store creation: DefaultTriples, Quads, DependencyGraph, SHACL and SKOLEM. The rdfox:TT SPARQL extension for referencing tuple tables has changed to reflect the new type of tuple table names. See Section 9.4.

  • Axiom management is now authorized using the resource specifier |datastores|{datastore-name}|axioms.

  • The RDF namespace used for system components that have IRIs has changed from http://oxfordsemantic.tech/RDFox# to https://rdfox.com/vocabulary# and the default prefix rdfox: has been updated to reflect this.

  • The compact operation now accepts a boolean option indicating whether redundant files should be deleted to reclaim disk space.

  • The APIs for granting and revoking privileges or memberships no longer produce errors if the system is already in the desired state. This was also the case in v5.6 and earlier however these operations now return a boolean indicating whether a change was made or not.

  • When setting server properties via REST, the operation request parameter must now be specified with value set.

  • Fixed a bug in the usage of the SHACL and DependencyGraph tuple tables in complex SPARQL queries.

  • SPARQL FROM statements now compute the set union of triples in the selected named graphs, as per the specification, instead of the multiset union.

  • The data store and in-memory tuple table parameters init-triple-capacity and max-triple-capacity have been replaced with init-tuple-capacity and max-tuple-capacity respectively.

  • The default library names for the libcrypto and libssl dynamic libraries on macOS have been updated from libcrypto.dylib to libcrypto.1.1.dylib and from libssl.dylib to libssl.1.1.dylib respectively.

  • Improvements have been made to the syntax highlighting and autocompletion of keywords in the Console’s “SPARQL” Mode.

  • Bug fixes to the Console’s “Explore” mode include a fix to the layout controls and Reset button to address an issue where a second automated graph layout could not be initiated.

  • The lowest supported version of Windows is now Windows 10.

  • The lowest support versions of Linux are Ubuntu 18.04 and Amazon Linux 2.

  • The JSON format for serializing explanations has been revised.

  • The following changes have been made to the oxfordsemantic/rdfox-init Docker image:

    • Server parameters can now be specified in a file instead of via the command line only.

    • The persist-roles or persist-ds server parameters are no longer defaulted to file and must be explicitly specified (either in the command arguments or in a parameters file).

    • The images no longer invoke RDFox when the server directory has already been initialized. Because of this, initialize.rdfox scripts are no longer executed with the continue-if-exists error policy.

Release 5.7 (10th August 2022)

  • Updated the JSON explanation format to serialise facts as objects, rather than strings.

  • Improved the memory usage of main memory tuple tables.

  • Support for the SecureTransport framework on macOS (previously selected with parameter value secure-transport) was dropped. OpenSSL is now required to use TLS on macOS.

  • Removed the clear facts-keep-rules operation.

  • Provided the APIs for changing the maximum allowed memory use of the server at runtime, without having to restart the server.

  • Performance improvements and bug fixes to the Console.

  • The following changes have been made to the configuration of RDFox servers:

    • When a server is created without an explicit server-directory parameter, no server directory will be used. In previous versions the server used a default location in the OS user’s home directory when the parameter was not specified explicitly.

    • In addition to the existing mechanisms for supplying them, server parameters may now be loaded from a file in the server directory. See Section 4.3.1 for details.

    • The shell, daemon and sandbox modes of the RDFox executable no longer have any associated defaults for the persist-roles or persist-ds server parameters. Instead, shell and daemon modes default the server-directory parameter to the same location as was previously used when this parameter was not set explicitly.

  • The modification of access control settings has been decoupled from the modification of server content. This has the following consequences:

    • Roles are no longer automatically granted privileges over the resources that they create.

    • Privileges specific to a resource (such as a data store, data source, tuple table or role) are no longer deleted when the resource is deleted. This, incidentally, fixes an issue introduced in RDFox v5.5 which prevented privileges specific to a data store persisting across sessions (RD-1306).

  • The sandbox mode of the RDFox executable now sets server parameter sandbox-directory to the empty string to disable file system sandboxing.

Release 5.6 (13th May 2022)

  • Added the getAnswerVariableName() method to the Cursor interface in JRDFox.

  • Fixed a bug where anonymous variables (called __AVAR__n) introduced internally during compilation would be returned in SELECT * queries (RD-1242).

  • Parameters specifying durations may now be specified with an optional unit of either s or ms. If no unit is specified, the integer part is interpreted as a number of whole seconds.

  • Introduced an EXPERIMENTAL extension of the REST API that allows remote execution of shell commands and an accompanying new mode for the RDFox executable, remote.

  • Added shell command rwtest to test reading and writing in data stores.

  • Improved the performance of reasoning with rules that reference the rdfox:SKOLEM built-in tuple.

  • Fixed a bug with the creation of data stores in the Console. If the server is configured to persist data stores to files by default, an option in the Advanced Settings dialog is provided to prevent a newly created data store from being persisted. This setting had no effect.

  • Improvements to the Console’s “Explore” mode:

    • A new “Highlight reasoning” control has been added to the toolbar. Relationships between entities that only exist as a consequence of rules may be displayed in a different colour to those relationships that are explicit within the RDF data.

    • The context menu on nodes has been enhanced with a control to copy the associated RDF entity.

    • A new context menu has been added to edges, with controls to delete the edge, copy the IRI of the edge’s predicate, or copy the associated fact.

    • The zoom and pan controls have been moved to the bottom right corner of the canvas.

    • The performance of visualizing small subgraphs of very large data stores has been improved.

  • Fixed the following issues in the Console’s “SPARQL” mode:

    • The editor inserted spaces in place of Tab characters inconsistently.

    • The line and column number of a SPARQL error were omitted from the error message.

  • Fixed the following issues in the Console’s “Explore” mode:

    • A relationship between the same pair of entities in multiple named graphs was incorrectly visualized.

    • A warning (that the visualized graph may be inconsistent with the current state of the RDFox data store) was incorrectly dismissed.

    • On visualizing a set of query results consisting of over one hundred individuals, the set of connections between the visualized nodes may have been incomplete.

    • Edges representing blank nodes (in generalized RDF data stores where blank nodes may be used as predicates) were labelled incorrectly.

    • Long string literals, particularly those containing a language tag, were incorrectly visualized in some browsers.

Release 5.5 (7th March 2022)

  • Introduced the following changes to the monitoring of importation, querying and reasoning in the CLI:

    • Added an import monitor called progress, which outputs information at the specified log-frequency.

    • Renamed the stats query monitor to profile and changed it to print information at the specified log-frequency.

    • Added a reasoning monitor called profile, which (periodically) outputs reasoning statistics on per rule or rule plan basis.

  • Fixed an issue with the handling of XSDDateTime and XSDDuration values.

  • Fixed an issue with escaping of whitespaces in IRIs (RD-1213).

  • Fixed a bug that could cause the endpoint to crash when a request is aborted with an error (e.g., due to timeout) and another request is issued afterwards.

  • Fixed a bug that prevented tuple tables/named graphs from being deleted.

  • Fixed a bug in the evaluation of property paths for which the source and target need to match.

  • Fixed a bug that could cause MIN and MAX aggregate functions to yield undeterministic behavior when applied to IRIs.

  • Added command unset to unset shell variables.

  • Added shell variable on-error to configure the behavior of the shell when an error is encountered.

  • Companion Docker images now return a non-zero return code if an error is encountered while running any mounted initialize.rdfox script, except if the error is due to an attempt to create a resource with a name that is already in use.

  • The proprietary application/x.sparql-results+json-abbrev and application/x.sparql-results+resourceid formats for SPARQL results were modified.

  • Fixed a bug with the TopK iterator.

  • Extended the endpoint with support for explaining reasoning.

  • Changed the available fact domains to explicit, derived, and all.

  • Provided a proprietary extension of SPARQL that allows finding out whether a fact is explicitly given in the input or is derived by a rule.

  • Changed the auto-update-statistics data store option.

  • Added the sandbox-directory server option.

  • Fixed a bug in the initialization of data source tuple tables for delimitedFile sources.

  • Improved the performance of the Console’s visualization of query answers.

  • Improved the Console’s visualization and exploration of large graphs of query answers and of large data stores.

  • Improved the Console’s exploration of reflexive properties.

  • Improved display of progress and error states in the Console’s “Explore” mode.

  • Fixed a bug in the Console editor that caused some SPARQL keywords within string literals to be interpreted as commands.

  • The JRDFox methods of the tech.oxfordsemantic.jrdfox.client.DataStoreConnection with names beginning getTransactionMust and setTransactionMust were renamed to instead begin getNextOperationMust and setNextOperationMust respectively.

  • The JRDFox method tech.oxfordsemantic.jrdfox.client.DataStoreConnection.getLastTransactionDataStoreVersion was renamed tech.oxfordsemantic.jrdfox.client.DataStoreConnection.getDataStoreVersionAfterLastOperation.

  • The HTTP method required for opening and advancing cursors was changed from POST to PATCH.

  • ETags are no longer sent while read/write transactions are open on a data store and the If-Match and If-None-Match headers can no longer be used in combination with a data store connection while a read/write transaction is open on the connection.

  • The version of a data store now changes only when a non-empty read/write transaction is committed. Previously, the version was incremented when any read/write transaction was begun.

Release 5.4 (25th October 2021)

  • Fixed bug that prevented query-time-limit and allow-query-time-limit-override from being modified when launching daemon processes.

  • Fix bug that caused process error messages not to be written to standard error.

Release 5.3 (21st October 2021)

  • Moved to a two-part version number format.

  • Replaced server parameter log-calls with api-log which has the same effect.

  • Added server parameter api-log.directory to configure the root directory for API logs.

  • Implemented recording of inputs imported by the system as an extension of the existing API logging feature. Recording can be limited or disabled with new parameter api-log.input-recording-limit.

  • API logging now captures details of exceptions and operation durations to the log.

  • Added a prefixes command to support listing and clearing the shell’s prefixes as well as restoring the default prefixes.

  • Added a proprietary MUL aggregate function that multiplies its arguments.

  • The web console is no longer EXPERIMENTAL.

  • Fixed the spelling of JRDFox API ConnectionFactory.getNumberOfLocaServerRoles to ConnectionFactory.getNumberOfLocalServerRoles.

  • Added new endpoint parameters query-time-limit and allow-query-time-limit-override.

Release 5.2.1 (29th June 2021)

  • Fixed a crash when importing malformed imports.

Release 5.2.0 (24th June 2021)

  • Introduced support for using-graph-uri and using-named-graph-uri when evaluating updates in the SPARQL endpoint.

  • Changed the keep-alive-timeout endpoint option to connection-keep-alive-time in order to unify this with a similar option for the Solr data source.

  • Added support for property paths in SHACL validation. Integrated SHACL validation with equality reasoning. Added an optional column to the built-in table rdfox:SHACL that specifies the fact domain of the validated data.

  • Fixed a bug with large incremental additions and deletions of rules (RD-1069).

  • Fixed a bug where property paths would be evaluated incorrectly in queries containing two or more FROM clauses (RD-1091).

  • Upgraded the import command to accept IRIs as well as file paths.

  • Ensure that relative IRIs entered into the shell are expanded against the shell’s base IRI.

  • Added a base command to print or set the shell’s base IRI.

  • Added the tuple table rdfox:DependencyGraph, which can be used to analyze the dependency graphs of Datalog programs.

Release 5.1.0 (7th May 2021)

  • Modified handling of aggregate functions with empty GROUP BY as described in this blog post.

  • Server and data store connections used by the RDFox shell can now be managed explicitly via the new commands srvconn and dsconn.

  • As part of the above change, the role switch command was removed.

  • Added new shell command evaluate for evaluating general SPARQL statements where a statement is either an update or a query.

  • Fixed a bug where the answer command would accept SPARQL updates as well as queries. The command now accepts only queries.

  • Fixed a bug which prevented update commands using the ! symbol from executing.

  • Fixed a bug where destructive operations in the shell would be executed even if the user did not confirm them (SUP-123).

  • The methods for saving a data store in binary format were moved from tech.oxfordsemantic.jrdfox.client.ServerConnection to tech.oxfordsemantic.jrdfox.client.DataStoreConnection.

  • Correctness fix to reasoning in a rare multithreaded situation (SUP-128).

  • Added new endpoint parameters channel-timeout, keep-alive-timeout and max-idle-connections.

  • The endpoint parameters sweep-period and sweeps-to-reclaim were replaced with the single parameter object-keep-alive-time.

Release 5.0.0 (24th March 2021)

  • Added native support for Apple Silicon.

  • Implemented support for SPARQL 1.1 property paths.

  • Upgraded delimitedFile and solr data source tuple tables to use the data types inferred from the source tables, where appropriate.

  • Renamed the ODBC data source parameter name-quote to quoted-identifier-quote.

  • Added HTTP request logging for the RDFox endpoint controlled by new endpoint parameters request-logger and elf-logger-fields.

  • Introduced the built-in table rdfox:SHACL for performing RDF constraint validation with the language SHACL.

  • Reworked the OWL support by removing the owl-in-rdf-support option and adding the explicit APIs for importing OWL axioms into a named graph (RD-879).

  • Added an EXPERIMENTAL C API. Header files are located within the include directory within the RDFox distribution.

  • A bug was fixed where the specified default graph was not always taken into account during importation of RDF data.

  • Improved the performance of answering queries that combine LIMIT and ORDER BY to compute the top K results.

  • Improved the evaluation of queries that combine ORDER BY and DISTINCT (SUP 118).

Release 4.2.0 (20th January 2021)

  • Integration with AWS Marketplace.

  • Images in the oxfordsemantic/rdfox-init repository now accept a subset of RDFox’s supported command line parameters to use when initializing new server directories.

  • Add a health check to the REST API.

  • Shell command dstore create now returns an error if there is an existing data store with the specified name.

  • Replace the shell commands load and save with corresponding subcommands to the dstore command.

  • Delete previously deprecated shell commands drop, explicate, init and updatestats.

  • Fix bug where HTTPS server certificates were not verified. This affected SPARQL LOAD and solr data sources.

  • Add verify-server-certificates parameter for solr data sources.

  • The shell command clear, as well as the delete subcommand of the dstore, role and tupletable commands, now require the user to confirm the operation unless force is specified as the final argument to the command.

  • Corrected a bug in the planning of rules where valid rules were rejected (RD-944).

Release 4.1.0 (30th November 2020)

  • The CONSTRAINT_VIOLATION built-in was removed.

  • A bug in the XML and JSON query answer formats was corrected where undefined variable would be serialized as UNDEF (instead of simply omitting their value).

  • The SKOLEM built-in function was removed, and instead the rdfox:SKOLEM built-in tuple table was introduced.

  • The internal:triple tuple table was renamed into rdfox:DefaultTriples.

  • All of SPARQL 1.1 Update has been implemented (i.e., LOAD, CREATE, DROP, etc.).

  • Added a number of built-in operators and functions, particularly numeric and date/time computation functions. For example, dates can now be subtracted to obtain a duration.

Release 4.0.0 (2nd October 2020)

  • Added a data source type solr for integrating with Apache Solr.

  • Added the option to treat negation in SWRL as nonmonotonic negation.

  • Validate parameters passed after the daemon mode specifier in an RDFox executable command.

  • Add a new proprietary built-in, CONSTRAINT_VIOLATION, to make blank node identifiers for constraint violations.

  • The default setting for the endpoint parameter min-secure-protocol is now tls12.

  • Corrected a bug in the reasoning algorithms where incrementally adding rules would sometimes produce incorrect results (RD-858).

  • Restoring persisted data stores uses multiple threads and can be significantly faster than previous versions of RDFox.

  • Persisted data store file format uses compression to reduce disk usage.

  • Internal rules to transitively close class hierarchies have been updated to exclude blank nodes (SUP-95).

  • Explicating facts is disallowed in combination with OWL support as that would keep increasing the set of axioms in the store (SUP-95).

  • Named graphs can now be updated through DELETE/INSERT queries.

  • The proprietary syntax for referencing tuple tables in SPARQL queries was changed (e.g., atom that used to be written as :A(?X,?Y) now needs to be written as TT :A { ?X ?Y }).

  • The license can now be passed in using environment variables RDFOX_LICENSE_CONTENT and RDFOX_LICENSE_FILE.

  • Add new shell command daemon. This permanently switches RDFox into daemon mode.

  • The import.rename-blank-nodes data store option was removed, and import.rename-user-blank-nodes data store option was added.

  • Adds an additional mode to the EXPERIMENTAL web console. The connections between the entities in a data store may be visually explored, with either a single individual or the result of a query as a starting point.

  • Added Transcribe command allow server state to be saved and potentially imported into another instance.

  • Added -temp-role command line option.

  • Roles can be created using a password hash. The password hash of existing role can be obtained programmatically or via the shell.

  • Allow forward slashes to be used for file paths in Windows.

  • Fixed a problem that could sometimes cause a crash when evaluating FILTER and BIND expressions that compare integers and floating point numbers (SUP-96).

  • Fixed a problem on Windows where one could start two endpoint processes on the same port (RD-592).

Release 3.1.1 (27th July 2020)

  • Fix bug where RDFox fails to start with a “unable to locate incompleted replace information” error on Windows.

Release 3.1.0 (7th July 2020)

  • Fixed a bug where RDFox would crash when a data store is restored from binary files and another operation (on a different data store) is running in parallel

  • Fix bug where RDFox would crash when the server was configured with data store persistence, but a data store without persistence is deleted (SUP-83)

  • Fix bug where RDFox would crash if the NOW() built-in function was written in lowercase (RD-777)

  • Fix bug where blank nodes were serialized incorrectly in the N-Triples and N-Quads formats (SUP-81)

  • Add a mechanism to constrain data store content using Datalog.

  • Add support for SWRL.

Release 3.0.1 (12th May 2020)

  • Fix bug where RDFox fails to start with a file in use error on Windows.

  • Fix bug that causes RDFox to end if the terminal window is resized or a screen session is reconnected during login prompt.

  • Fix bug where data store snapshot files are not deleted when data stores are deleted.

  • Fix bug that caused RDFox to hang or crash if an attempt is made to import invalid data.

  • Allow Ctrl+C to interrupt import operations.

  • Allow Ctrl+C to interrupt loading of persisted data stores during restart.

Release 3.0.0 (7th May 2020)

  • The term ‘user’ is replaced by the term ‘role’ in all JRDFox APIs related to authentication. For example tech.oxfordsemantic.jrdfox.client.DataStoreConnection.getUserName() is replaced by tech.oxfordsemantic.jrdfox.client.DataStoreConnection.getRoleName().

  • When creating server connections for the first time via JRDFox APIs, the default role name and password are now “guest” and “guest”.

  • All references to the namespace org.semanticweb.owlapi have been removed from the JRDFox code base and the owlapi-*.jar file is no longer included in the RDFox distribution. Consequently the following features are no longer available:

    • the overload of tech.oxfordsemantic.jrdfox.client.DataStoreConnection.importData that accepted an org.semanticweb.owlapi.model.OWLOntology

    • the command line facility to generate a Datalog program from an OWL ontology

Release 2.1.1 (14th November 2019)

  • Correct the behavior of COUNT(?v) and COUNT(*).

Release 2.1.0 (18th October 2019)

  • owl-in-rdf-support is no longer EXPERIMENTAL.

  • The JRDFox features that make use of owlapi-*.jar (one of the DataStoreConnection importData methods, and the GetRules utility) are now deprecated. This dependency will be removed in a future release of RDFox. Import of OWL documents (in Functional Style and Turtle syntaxes) will continue to be supported by the remaining importData methods.

Release 2.0.1 (23rd September 2019)

  • Fix bug where a crash may occur when the Java or REST interfaces are used to import large data sets (RD-531).

Release 2.0.0 (4th September 2019)

  • The MIME types for the RDFox-proprietary SPARQL answer formats have been consolidated to us the x. subtype tree

  • New proprietary SPARQL answer formats have been added that abbreviate IRI prefixes in the result

  • The binary answer format has been changed

  • Renamed the parameters for obtaining diagnostic information in the REST API

  • Added calls to all APIs for retrieving server-level diagnostic information

  • Removed the query domain parameter in all APIs for exporting data and added the ability to pass in arbitrary key-value pairs

  • Adapted the syntax of the export command to support the change from the previous item

  • Optimized the shell interface by not requiring file names to be enclosed in quotes

  • Embedded an experimental HTML application which is served from /console when the endpoint is running

  • Only add the Access-Control-Allow-Origin header to HTTP responses if the user has configured the allowed origin

  • Added support for importing from files containing axioms represented in OWL 2 functional-style syntax.

  • Added experimental support for OWL 2 axioms translated from imported RDF data.

  • SPARQL query evaluation has been reworked to improve performance and eliminate errors in the returned solution for a small number of query types.

Release 1.6.0 (15th May 2019)

  • RDFox and JRDFox now require a license file

  • Improved support for xsd:decimal on Windows

Release 1.5.0 (17th April 2019)

  • RESTful API supports the HTTP X-HTTP-Method-Override request parameter to allow HTTP PATCH requests to be made from client code that does not directly support PATCH (e.g. Java HttpURLConnection)

  • Improved support for xsd:decimal on macOS and Linux

Release 1.4.0 (20th March 2019)

  • Added support for the following SPARQL 1.1 functions:

    • NOW()

    • UUID()

    • STRUUID()

    • RAND()

    • MD5()

    • SHA1()

    • SHA256()

    • SHA384()

    • SHA512()

    • IRI()

  • RDFox now retains the datatypes of literals in the input — that is, xsd:int is no longer normalized to xsd:integer, and similarly for other derived datatypes (RD-194)

  • Added initial support for xsd:decimal (RD-162)

  • Performance improvements in the evaluation of most built-in functions (RD-196)

  • Import options rename-blank-nodes-on-import and convert-unknown-datatypes-to-strings were replaced with options import.rename-blank-nodes and import.invalid-literal-policy described in Section 5.4.1

  • Shell variables containing backslash characters are now correctly expanded on the command line (RD-219)

  • There is no restriction on the number of files that can be imported on a single line in the shell (RD-89)

Release 1.3.3 (8th February 2019)

  • SSL/TLS for the RESTful API using openSSL (Windows, macOS, Linux) and Secure Transport (macOS only)

  • Fixed a bug where sending Content-Type: 0 with DELETE requests was incorrectly detected as having a body

  • Extracted the data store statistics into separate data structures that can be administered using the various APIs (RD-27); thus, there is now a way to update the statistics using the REST API

  • Repurposed the stats shell command to manage statistics, and deprecated the updatestats command in favor of stats update

  • Introduced the info shell command that merges the functionality of the old stats command, the rulestats command, and the ruleplan command; added the option to print summary of the information about a data store; removed the rulestats and ruleplan commands

  • Fixed a bug where tuple tables imported from a data source could have repeated column names

  • Extended the OWL translator to support datatypes and datatype restrictions according to the OWL 2 RL specification (RD-53)

  • Improved handling of out-of-memory exceptions (RD-50)

  • Improved handling of situations when the capacity of a data store is exceeded (RD-90)

  • Fixed a bug where parallel import of Turtle files would sometimes slow down or even crash after loading some number of files

  • Improved the performance of importing large numbers of xsd:float and xsd:double values

  • Fixed the bug where incrementally added rules sometimes were not handled correctly (RD-120)

  • OFFSET and LIMIT can not occur in a SPARQL query in an arbitrary order

  • The RDFox command line supports common line editing functions including arrow-key navigation and command history on platforms where this was previously unavailable (macOS and Linux)

  • Commands used when starting RDFox have been revised (now shell or daemon) and options have been added to enable logging of API calls and to set a maximum allowable memory size.

  • Added support for cryptographic hash functions MD5, SHA1, SHA256, SHA384 and SHA512 in rules and queries

  • UCASE() and LCASE() functions are Unicode aware and use the unconditional case mappings as defined by the Unicode Standard 11.0 (RD-8)

  • Added the REGEX() and REPLACE() SPARQL 1.1 functions (RD-5)