Next Release to Support Automatic Loading of Industry Standard Schemas

Currently, Gepsio parses an XBRL instance and automatically loads any schemas referenced by a <schemaRef> element. The facts in the XBRL instance are then automatically validated against the loaded schemas. However, Gepsio does not automatically load any industry-standard schemas not explicitly referenced by the <schemaRef> element. Since XBRL instances do not typically make an explicit reference to industry-standard schemas (nor should they), facts that reference those schemas cannot be validated.

Consider, for example, the XBRL instance at http://xbrlsite.com/US-GAAP/BasicExample/2010-09-30/abc-20101231.xml. It contains one <schemaRef> element so that a linkbase can be loaded. However, the facts in this instance reference namespaces such as http://xbrl.us/dei/2009-01-31 and http://xbrl.us/us-gaap/2009-01-31. Since there is no schema reference for those namespaces, the facts that use those namespaces fail validation. The trick, though, is that those namespaces reference well-known industry standard namespaces that identify well-known schemas with schema source at well-known locations across the Internet.

The next version of Gepsio will add support for these well-known namespaces and will automatically load schemas referenced by the XBRL instance’s <xbrl xmlns:...> syntax even though the schema locations for the schemas are not explicitly referenced by a <schemaRef> element in the instance. Doing this will allow the facts in the instance using those namespaces to validate with full schema fidelity.

As of this writing, Gepsio supports the following industry-standard schemas:

  • Document Information and Entity Information 2009
  • US-GAAP 2009

Support for many other industry-standard schemas will be added as testing continues. The important note is that the code now has the infrastructure in place to support this notion, and support for other industry-standard schemas can be added relatively easily, and in one place, now that the infrastructure is available.

This change is in the develop branch in GitHub. The changes are available in this commit and will be available in the next update to Gepsio’s NuGet package.

Advertisements

Gepsio Repository Now Available on GitHub

The Gepsio code base has a new home on GitHub! The latest code base, built for .NET Standard 2.0, is available at https://github.com/JeffFerguson/gepsio. Feel free to clone it and take a look!

Please keep in mind that the Gepsio Git repo is in preview. The solution in the repo is based on .NET Standard 2.0 and requires Visual Studio 2017 Preview 15.3.0 Preview 3.0 or higher, as well as .NET Core 2.0 Preview 2 or higher, to build. If you need code compatible with the current shipping production releases of Visual Studio 2017 and the full .NET Framework, please see Gepsio’s legacy Codeplex repository.

Gepsio and .NET Core 2.0 Preview 2

Microsoft recently announced .NET Core 2.0 Preview 2 and, in keeping with the Gepsio development roadmap for 2017, work continued to see how the Gepsio code base would survive the move from the .NET Framework to .NET Core. A previous blog post discussed this same work with .NET Core 2.0 Preview 1.

Spoiler alert.

The unit tests in the .NET Standard-based Gepsio solution have been updated to use the latest available version of the XBRL Conformance Suite (XBRL-CONF-2014-12-10). The solution builds a .NET Standard 2.0 build of Gepsio and a .NET Core 2.0 build of the unit tests, and the tests run to the same point at which the older .NET Framework build of Gepsio runs. The tests seem to run a bit more slowly than they do in the .NET Framework, but, since .NET Standard 2.0 is in preview mode, it can be assumed that some performance tuning has yet to be done.

This is fantastic news! Gepsio is on its way to being able to provide services to a variety of Windows and non-Windows platforms, extending its reach beyond .NET to a variety of operating systems and device platforms.

Next Steps

After doing some final cleanup, the .NET Standard-base Gepsio solution will be moved up to GitHub. A separate blog post will be created when the new repo is ready. The older .NET Framework-based Gepsio solution will remain on Codeplex.

It is important to note that the solution that will be posted to GitHub is based on .NET Standard 2.0 and requires Visual Studio 2017 Preview 15.3.0 Preview 3.0 or higher, as well as .NET Core 2.0 Preview 2 or higher, to build. If you need code compatible with the current shipping production releases of Visual Studio and the .NET Framework, continue to reference Gepsio’s existing Codeplex repository.

Gepsio and .NET Core 2.0 Preview 1

Microsoft recently announced .NET Core 2.0 Preview 1 and, in keeping with the Gepsio development roadmap for 2017, work began to see how the Gepsio code base would survive the move from the .NET Framework to .NET Core. Good news and bad news came out of this work:

  • Good News: The code compiled with without changing a line of code!
  • Bad News: A bug in .NET Core 2.0 Preview 1 will prevent Gepsio from running under that particular preview release.

The Good News

Building Gepsio under .NET Core 2.0 Preview 1 was a breeze (although it needs to be built under a preview version of Visual Studio). Since all of the classes used in the Gepsio code base are available in .NET Standard 2.0, all of the code was able to build with no changes. Excellent!

The Bad News

After Gepsio built under .NET Core 2.0 Preview 1, the time came to run the unit tests. Unfortunately, the tests failed right away (cue the sad trombone). Running the very first test in the XBRL Conformance Suite under .NET Core 2.0 Preview 1 throws the following exception:

System.Xml.Schema.XmlSchemaException occurred
HResult=0x80131941
Message=Type 'http://www.xbrl.org/2003/instance:monetaryItemType' is not declared.
Source=<Cannot evaluate the exception source>
StackTrace:
at System.Xml.Schema.XmlSchemaSet.InternalValidationCallback(Object sender, ValidationEventArgs e)
at System.Xml.Schema.BaseProcessor.SendValidationEvent(XmlSchemaException e, XmlSeverityType severity)
at System.Xml.Schema.BaseProcessor.SendValidationEvent(XmlSchemaException e)
at System.Xml.Schema.Compiler.CompileElement(XmlSchemaElement xe)
at System.Xml.Schema.Compiler.Compile()
at System.Xml.Schema.Compiler.Execute(XmlSchemaSet schemaSet, SchemaInfo schemaCompiledInfo)
at System.Xml.Schema.XmlSchemaSet.Compile()
at JeffFerguson.Gepsio.Xml.Implementation.SystemXml.SchemaSet.Compile() in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\Xml\Implementations\SystemXml\SchemaSet.cs:line 83
at JeffFerguson.Gepsio.XbrlSchema..ctor(XbrlFragment ContainingXbrlFragment, String SchemaFilename, String BaseDirectory) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlSchema.cs:line 169
at JeffFerguson.Gepsio.XbrlFragment.ProcessSchemaNamespaceAndLocation(String schemaNamespace, String schemaLocation) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlFragment.cs:line 374
at JeffFerguson.Gepsio.XbrlFragment.ProcessSchemaLocationAttributeValue(String schemaLocationAttributeValue) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlFragment.cs:line 368
at JeffFerguson.Gepsio.XbrlFragment.ReadSchemaLocationAttributes() in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlFragment.cs:line 348
at JeffFerguson.Gepsio.XbrlFragment..ctor(XbrlDocument ParentDocument, INamespaceManager namespaceManager, INode XbrlRootNode) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlFragment.cs:line 160
at JeffFerguson.Gepsio.XbrlDocument.Parse(IDocument doc) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlDocument.cs:line 275
at JeffFerguson.Gepsio.XbrlDocument.Load(String Filename) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio\XbrlDocument.cs:line 179
at JeffFerguson.Test.Gepsio.XbrlConformanceTest.ExecuteVariation(String TestcaseXmlSourceDirectory, XmlNode VariationNode) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio.Test\XbrlConformanceTest.cs:line 97
at JeffFerguson.Test.Gepsio.XbrlConformanceTest.ExecuteTestcase(String ConformanceXmlSourcePath, XmlNode TestcaseNode) in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio.Test\XbrlConformanceTest.cs:line 76
at JeffFerguson.Test.Gepsio.XbrlConformanceTest.ExecuteXBRLCONF20141210Testcases() in C:\Users\jefff\Source\Repos\gepsio\JeffFerguson.Gepsio.Test\XbrlConformanceTest.cs:line 44

The test uses the following schema:

<!-- edited by Masatomo Goto (Fujitsu) -->
<!-- XBRL 2.1 Tests -->
<!-- Copyright 2003 XBRL International. See www.xbrl.org/legal. All Rights Reserved. -->
<schema targetNamespace="http://example.com/xbrl/taxonomy" xmlns="http://www.w3.org/2001/XMLSchema" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xbrli="http://www.xbrl.org/2003/instance" xmlns:link="http://www.xbrl.org/2003/linkbase" xmlns:xlink="http://www.w3.org/1999/xlink">
<import namespace="http://www.xbrl.org/2003/instance" schemaLocation="http://www.xbrl.org/2003/xbrl-instance-2003-12-31.xsd"/>
<element name="changeInRetainedEarnings" type="xbrli:monetaryItemType" substitutionGroup="xbrli:item" id="a1" xbrli:periodType="duration"/>
<element name="fixedAssets" type="xbrli:monetaryItemType" substitutionGroup="xbrli:item" id="a2" xbrli:balance="debit" xbrli:periodType="instant"/>
</schema>

In this schema, the changeInRetainedEarnings element is of type xbrli:monetaryItemType, which is defined in the imported schema. The imported schema’s types were not being picked up by .NET Core 2.0 during the schema set compilation process as they are in the full .NET Framework.

The issue was raised on GitHub and was traced to a bug in the .NET Core 2.0 Preview 1 build. The issue, which has its roots in this bug, has been fixed but was fixed too late to be included in .NET Core 2.0 Preview 1.

Next Steps

The next steps for Gepsio is to wait for .NET Core 2.0 Preview 2 to be released and to retry the compilation and unit testing process. Perhaps Preview 2 will allow Gepsio to get farther along in the .NET Core runtime.

Codeplex Shutting Down, But Gepsio Continuing on GitHub

Microsoft has announced that the Codeplex site will be retired this year. This news does not affect Gepsio in any way. As noted in the original roadmap plan for Gepsio, as well as the March 2017 update to that plan, Gepsio will be moving from Codeplex to GitHub in 2017.

The anticipated shutdown date for Codeplex is December 15, 2017. Gepsio’s 2017 development plan notes that Gepsio will move over to GitHub when .NET Standard 2.0 is released. Since the move to GitHub is planned during the first public preview release of .NET Standard 2.0, and since Microsoft’s current roadmap for .NET Standard lists a public preview release of .NET Standard 2.0 in the second quarter of 2017, Gepsio’s code base and related materials should be moved over to GitHub well before the December 15 shutdown date for Codeplex. If there is a need to do so, Gepsio could be moved over to GitHub earlier than that; however, with all of the other migration activities going on as well as Gepsio’s move from .NET Framework to .NET Standard, it made the most sense to make a “clean cut” over to GitHub once the code’s migration to .NET Standard is complete.

The Gepsio project thanks the Codeplex team at Microsoft for offering a home to the Gepsio source since 2008.

March Update on Development Roadmap for 2017

Microsoft has released Visual Studio 2017, which is the development environment used for Gepsio development. Gepsio’s development roadmap for 2017, published in December 2016 and outlined here, noted that Gepsio would be moving to .NET Standard 2.0 once Visual Studio 2017 was released, as .NET Standard 2.0 would allow Gepsio to support a variety of environments, including the full .NET Framework, .NET Core, Xamarin.iOS, Xamarin.Android, and Linux.

The original roadmap was written on the assumption that .NET Standard 2.0 would be shipping “out of the box” with Visual Studio 2017. According to Microsoft’s .NET Core Roadmap, .NET Standard 2.0 and .NET Core 2.0 will not be released until Q3 2017, with public preview builds arriving in Q2 2017. Therefore, the project will remain in Codeplex and Visual Studio 2015 until the public previews of .NET Standard 2.0 are available.

To recap, once .NET Standard 2.0 becomes available, the project will move forward as follows:

  • the solution will migrate from Visual Studio 2015 to Visual Studio 2017
  • the Gepsio assembly will migrate from a .NET Framework target to a .NET Standard 2.0 target
  • the Gepsio unit tests will migrate from a .NET Framework target to a .NET Core 2.0 target
  • the repository will migrate from Codeplex over to Github
  • the repository access will migrate from TFS to git

For now, the work will remain at Codeplex and built by Visual Studio 2015. There have been several improvements made lately, with changeset history visible here, and it’s high time for a new release. Perhaps a new build can be pushed out to NuGet in April.

Development Roadmap for 2017

Visual Studio 2017 and .NET Standard promise to bring exciting changes to .NET-based development efforts, and Gepsio will be taking full advantage of these changes. Gepsio’s cross-platform vision, as outlined here, is to make use of .NET Standard to bring Gepsio’s functionality to the full .NET Framework, .NET Core, the Universal Windows Platform, Mono, and Xamarin for iOS and Android.

The Wait for .NET Standard 2.0

The big waiting game for moving on the cross-platform story is with .NET Standard 2.0. As discussed previously, .NET Standard 2.0 will contain support for the classes in the XML Schema namespace not available in earlier versions of .NET Standard. Since Gepsio heavily leverages XML Schema, .NET Standard support for XML Schema is a prerequisite for any migration of the code base over to .NET Standard. The good news is that the .NET Portability Analyzer has been run against the current Gepsio binary and its API usage was found to be 100% compatible with .NET Standard 2.0. The wait for the 2.0 release is necessary, because the code base is only 73.31% compatible with .NET Standard 1.4 and 73.68% compatible with .NET Standard 1.5 and 1.6 due to Gepsio’s use of classes in the XML Schema namespace. It is currently assumed that .NET Standard 2.0 will be released in the same time frame as Visual Studio 2017, which, according to current documentation, is currently slated for some time in the spring of 2017.

From Codeplex to GitHub

Once .NET Standard 2.0 becomes available, and the code base is migrated to the new platform, the project will find a new home at GitHub. There have been a few requests to move the project over to GitHub to allow Git clients to repo the solution, and the new .NET Standard reconfiguration seems like a good time to make the change. The historical project will remain at Codeplex, but, once the .NET Standard migration is complete, new code will be available at GitHub.
As a part of this change over to Git, the solution folders will be simplified to match de facto conventions used by other GitHub projects. There will be a few folders at the root of the solution:
  • src, where the source will be stored
  • test, where the unit tests will be stored
  • docs, for documentation and possibly a GitHub Pages site for the project
  • nuget, for NuGet specification files and other related information

Simplified Solution Structure

A previous post discussed a restructuring of the solution which would enable Gepsio to provide several different versions of the .NET Framework. The restructuring, based on a Visual Studio 2015 project type called Shared Projects, allowed one code base to be used in different referencing projects, each targeting a different version of .NET. With the move to .NET Standard 2.0, only one version of the .NET Framework — version 4.6.1 — will be supported, so all of the other various .NET Framework target folders will be removed.
The overall concept of a shared project folder containing the source, as well as other projects for various targets, will remain. It is expected that the solution will have projects for various platform-specific compilation targets, such as .NET Core, Mono, and Xamarin, but they should all be able to use the code from the main Shared Project containing the Gepsio source.

UWP

A previous post outlined the original thoughts behind Gepsio’s support for the Universal Windows Platform. At the time, it was envisioned that the solution would contain a separate implementation of the various XML interfaces usable by the UWP compilation target. Work was well underway when .NET Standard was announced. However, .NET Standard promises to support UWP with a standard implementation, so UWP-specific interface implementations for XML support are unneeded. Therefore, the UWP project will be removed and replaced with a UWP compilation project that supports .NET Standard.

Increased XBRL Conformance

At the end of the day, Gepsio’s raison d’être is to provide support for the XBRL specification. To be honest, this work has been sidelined in favor of building up cross-platform support. Once .NET Standard 2.0 is in place, and Gepsio’s code base is set up to offer that cross-platform support, work can continue on Gepsio’s support for the XBEL specification. This vision includes support for the various XBRL specifications, including Dimensions and Formulas.