SIF enabled applications

[penfold_99]

Given that one of the goals of SIF is to allow interoperability of data between systems, it could be monopolistic for a company offer SIF as a (paid for add-on) thereby ensuring vendor lock-in. Is this the current state of affairs? I figured api's would be free in 2012?

 

The DfE framework allows schools to have free access to data but didn't cover updating information. This is still where the monopoly lies in the cost a supplier charges to write data back.

[penfold_99]

@CyberNerd - that's what i was thinking - imagine it's slow uptake, because saying you support it is good, actually supporting it - not good for business.

 

The only supplier that this would be affect is capita as its the only one to charge for writing data back. As SIF would allow multiple suppliers to write back but capita could only charge a fee once.

 

Really, it should be enforced, certainly for education / government funded areas (what does that mean these days ??). I wonder if it's part of the recent framework released http://www.edugeek.net/forums/mis-systems/76154-something-keep-eye-dfe.html#post807968

 

Surely a massive factor in changing a MIS/VLE will be the transfer of data and associated cost. It would be silly if the best value is obliterated by cost to move. I think mostly the receiving company bears the onus of moving data, but how much do they say they can move. The criteria for companies being on that list should have been contingent on SIF compliance and availability.

 

AS part of the framework suppliers had to agree to supporting the ISB standards even though they have not been released. SIF is working with the ISB about the practical application of these standards. In an upcoming release of SIF specification it will allow for multiple data models to be transferred between systems. SIF could adopted the ISB data model and use its transport layer to move it between systems.

[bantonia]

SIF sounds good according to the specification and I was particularly interested when I looked at the Abstract:

 

An alternative SOAP-based transport and corresponding set of WSDL files which allow web services to

fully participate in these interactions;

 

But was disappointed when I got to page 283 which stated:

 

To do so and to meet the final requirement of a Web Service per [WSARCH], the use of SOAP messages, would require redefinition of much of SIF using SOAP messages. The SIF Association's Web Services Task Force has determined that this exercise has little value currently, given SIF's precisely defined transport layer and installed base. The task force has left it as a future task how to best leverage Web Services in the future of SIF's infrastructure, if at all.

 

To not support SOAP and WSDL is probably going to limit the takeup of SIF as there are no tools to create interfaces for the programming language being used. Each vendor would have to start from scratch from the specification. Is there any possibility of web services via SOAP? It is a standard, whereas SIF, although well thought out, does not help its uptake by being just another xml based protocol to get ones head around in terms of programming from the bottom up. Currently I've gone my own way using SOAP to interface between SIMS and Moodle. To use SIF via SOAP and WSDL would be ideal but until that's available I'll stick to what I've got. Next part to do, SIMS to AD via SOAP using C#! I already run CommandReporter.exe via web services and SOAP so it's just the AD client part to do, the WSDL is already generated.

[vikpaw]

@bantonia - man, you're keen to have read 283 pages!

[bantonia]

Skipped a few pages in between as I wanted to get to the SOAP and WSDL like the specification was teasing!

 

Overall I agree with the ideals it sounds good but as a programmer I don't particularly want to start from scratch having to interpret accurately most of the 283 pages before I got to the disappointing statement. Currently there are, as far as I know, no open source libraries, certified or otherwise in any programming language to create SIF messages. I tried to install OpenZIS yesterday as a test site and eventually came to a brick wall. There were also comments on GitHub that OpenZIS is no longer developed, how accurate that is I personally do not know, I may be passing on mis-information. I would be very happy to be stand corrected. So with ZIT server not working so nothing to test against, I am reluctant to spend the time and effort to move away from something I already have working via SOAP albeit not to the same ideals as SIF. As for other vendors, everyone seems to be keeping their own developments regarding SIF very close to their chests as it is a commercially viable product. Open specification is one thing but open source is quite another factor which needs to be addressed to enable the proliferation in use of the specification.

[GrumbleDook]

To some extent I can understand why some companies produce an agent (or have commercial relationship with someone to do one) and you buy into that rather than give open access into the inner workings of their systems. To some extent it is saying "There is a way to get data out in an easy and standard way, it costs x amount and we know there is good dev and support for it. What you do with it once it hits the ZIS is your business and feel free to be as open source as you want with that bit".

[bantonia]

The thing is, I can currently do that with SOAP to pull data out of SIMS and store it in an intermediate database and use it as I wish http://www.beaconhillcott.net/ but to increase the use of SIF and ZIS would it not be better to empower programmers access to the interface by creating open source libraries or via WSDL which everyone can share. If you do want to put a price tag on your connection you could have a switch to turn that feature on if required. I've also begun creating access indirectly to Moodle via SOAP/WSDL using the REST library I created. With the intermediate glue between the two systems I'm able to keep users and classes up to date. Obviously I would like to use SIF but at present it means there would be a large development process to overcome which some already have covered.

[penfold_99]

@bantonia

 

Which document are you reading as SIF has support for SOAP and WSDL? I believe it was added in 2.4 or 2.5 of the infrastructure specification. I will double check.

 

We support the OpenZIS project in the UK and are currently using OpenZIS, what issues are you facing? OpenZIS only support the classic HTTP infterface and not SOAP.

We work closely with Chris Whitley the original developer, development has slowed recently as Chris is contracted else where.

 

Open specification is one thing but open source is quite another factor which needs to be addressed to enable the proliferation in use of the specification.

I disagree, use of the standard is not dependant on the end product being open source. The cost/benefit trade off needs to be in the interest of the supplier so they choose to adopt SIF.

If its not then irrespective of the products being open sourced supplier wont adopt it.

 

If you are developing for multiple MIS and require writeback then SIF is the only viable option to save money as you dont need to partner with all the MIS suppliers.

[bantonia]

SIF Version 2.5, extract in an earlier post states, but in my translation, "We're not going to bother with SOAP and WSDL as it's too much hassle". The problem is development time is going to be increased and errors introduced if each vendor creates their own flavour of the messaging system.

 

Anyway, it's all academic without access to a working ZIT to test against.

 

The sql scripts which had interesting spelling of "archive", for I think, 3 of the table names, which I hope is reflected in the code, these tables had been truncated removing the "archeve" (note the spelling) so duplicate tables were trying to be created. So got around that problem by adding on the incorrectly spelled extension to those tables.

 

After configuring Apache with 2 virtual sites using the information from Installation « OpenZIS UK, I know it's a different flavour of Linux, I use OpenSuSE, I eventually opened a browser to see a login box. I quick trawl through the database to find a username and password to find admin admin to eventually hit "Authentication server is unreachable". Looking at the Apache logs it's trying to find a file in the root of the ADMIN_SERVER directory called "index" however the only file there of any import is index.php. Ok, have a look through the .js files and find the code where the "index" reference is used and change it to "index.php". Good, got further, it says "Connecting Sending data". However it never completes. Looking in the "logs" directory in the .xml file created there it says "login is not a recognised .....", or something like that. Anyway, editing the beginning of the same .js file to by-pass the authentication I do get to a configuration interface.

 

So that is where I'm at. Until I get any further SIF is on the shelf.

 

P.S.

The error in the "recordshelf-log.xml" file was

Invalid controller specified (login)

 

and the file I was editing "mainExt.js" line 228

Edited April 23, 2012 by bantonia
Added P.S.

[bantonia]

It appears there is confusion within the 2.5 SIF documentation one states, extract from the first pdf on page http://specification.sifinfo.org/Implementation/us/2.5/:

 

To do so and to meet the final requirement of a Web Service per [WSARCH], the use of SOAP messages, would require redefinition of much of SIF using SOAP messages. The SIF Association's Web Services Task Force has determined that this exercise has little value currently, given SIF's precisely defined transport layer and installed base. The task force has left it as a future task how to best leverage Web Services in the future of SIF's infrastructure, if at all.

 

Whereas further down the same web page a link to a zipped archive of WSDL files.

 

Has someone not proof read the pdf or am I getting the wrong end of the stick?

 

Anyway, OpenZIS still non-functional.

[penfold_99]

It appears there is confusion within the 2.5 SIF documentation one states, extract from the first pdf on page http://specification.sifinfo.org/Implementation/us/2.5/:

 

To do so and to meet the final requirement of a Web Service per [WSARCH], the use of SOAP messages, would require redefinition of much of SIF using SOAP messages. The SIF Association's Web Services Task Force has determined that this exercise has little value currently, given SIF's precisely defined transport layer and installed base. The task force has left it as a future task how to best leverage Web Services in the future of SIF's infrastructure, if at all.

 

Whereas further down the same web page a link to a zipped archive of WSDL files.

 

Has someone not proof read the pdf or am I getting the wrong end of the stick?

 

I have put a request in the the SIF association for clarification on the documentation.

I believe SOAP and WSDL could be covered under the SIF Global Web Services Implementation Specification 2.5 and SIF Global Web Services Implementation 2.5 Developer’s Toolkit.

Once I have an answer I will post it here.

[bantonia]

Cheers.

 

I assume as this is fairly new that OpenZIS would not support SOAP and WSDL anyway although any new server needs to implement 2.5 and any older servers would need that function added.

 

It appears from the 59 page pdf document that servers will need to support both SIF and SOAP message types and therefore translate between the two.

[bantonia]

Finally got a working OpenZIS, thanks to the help of Christopher Whiteley.

 

Ok, can anyone let me know how do I get access to the SIMS SIF Agent, if there is one?

 

Or do I have to modify my current Web Service access into SIMS via CommandReporter.exe to be compatible with SIF?

Currently it uses SOAP calls, passing the command line to the Web Service and sends an array of data in a SOAP message back to the client, at least it would have read access.

 

Thanks in advance.

Edited April 26, 2012 by bantonia

[penfold_99]

@bantonia

 

I have had a reply back from the SIF association about the documentation query.

It was a case of the text being missed and not updated before being release and its now being updated. Many thanks for spotting this.

 

The section will now read.

C.9 SIF and Web Services

With the addition of Web Service support in the SIF v2.5 specification, it is now possible to write a “pure” SIF web service which:

• Supports a defined SIF Application WSDL interface for receiving asynchronous, Events, Requests and Response methods over the SOAP transport

• Acts as a web client of the a set of ZIS web services, which also support SIF-specified WSDL interfaces over the SOAP transport, and which together encapsulate the ZIS functionality

• Can seamlessly and transparently interoperate with other SIF applications, whether they are web services themselves, or pre-existing SIF v2.x HTTP/S based applications.

 

The details of this support are contained in the SIF Global Web Services Implementation Specification volume of this release.

 

The two transport layers are covered in the following documentations.

 

SIF Global Infrastructure Implementation Specification 2.5 = Classic HTTP Transport

SIF Global Web Services Implementation Specification 2.5 = SOAP Transport

 

The WSDL is included in the Developer Toolkit http://specification.sifassociation.org/Global/2.5/ws/SIFv2.5WebServiceDeveloperKit-3.zip and http://specification.sifassociation.org/Global/2.5/ws/SIF_ws2p5.pdf

[bantonia]

Sorry, just another little niggle.

 

I'm trying to access the 1.4 schema on http:/specification.sifassociation.org/implementation/UK/1.4/XSD/SIF_Message.xsd but it says "The page cannot be found".

 

I've found that NetBeans can create Java classes and artefacts through JAXB by reading the XSD schema file. Looking at the 2.5 schema, which is a US schema (I think), it almost works, there is a problem only parsing CalendarDateTime. Anyway, as the UK is currently version 1.4, where can a copy of this file be accessed apart from the above URL which does not work?

[bantonia]

Ignore my ramblings, found a zip file! Although the URL above does not work. JAXB in NetBeans created Java SIF source code without error!

[penfold_99]

Ignore my ramblings, found a zip file! Although the URL above does not work. JAXB in NetBeans created Java SIF source code without error!

 

It would be better to use 1.2 as its the version that has been implemented in the real world.

 

The open-adk project needs to catch up as that only supports 1.2. The adk or agent development kit has been used by most suppliers to support Sif in their application.

[bantonia]

1.2 works if you choose the single file non-annotated versions.

 

What is the difference between the SIF_Message and DataModel schemas?

Edited April 27, 2012 by bantonia

[penfold_99]

1.2 works if you choose the single file non-annotated versions.

 

What is the difference between the SIF_Message and DataModel schemas?

 

SIF_Message is the transportation layer schema, it covers how the sif agent talks to the zis.

 

The Data Model schema governs what data can be sent.

[bantonia]

Hi,

 

I created a SIF_Ping message, following the information and example SIF_Message from SIF Implementation Specification (United Kingdom) 1.2 - Messaging and SIF Implementation Specification (United Kingdom) 1.2 - Architecture. I have also created an Agent entry in the OpenZIS server and have used the SIF_SourceId in the message I'm trying to test. Looking at the contents sent through wireshark, all the headers are present and correct and the XML message itself follows the pattern in the example except in three places, the Version and xmlns attributes are swapped round, there is no space in the SIF_Ping (ie ) and there is an additional

 

Any clues would be appreciated.

 

I'm not using any security at present, there is no data being accessed. The OpenZIS server is set to 1.2.

 

Regards

Edited April 30, 2012 by bantonia

[bantonia]

Ahhhhhhhhh....... One thing, looking at the OpenZIS ZIT code, it is currently forced to authenticate either by certificate or password! I'll switch that off for testing and see what happens. Also it appears the XML requires a \n character between each group of open/close tags for the PHP to explode to help out in the validation process, I'll switch that off as well. Edited April 30, 2012 by bantonia

[penfold_99]

Hi,

 

I created a SIF_Ping message, following the information and example SIF_Message from SIF Implementation Specification (United Kingdom) 1.2 - Messaging and SIF Implementation Specification (United Kingdom) 1.2 - Architecture. I have also created an Agent entry in the OpenZIS server and have used the SIF_SourceId in the message I'm trying to test. Looking at the contents sent through wireshark, all the headers are present and correct and the XML message itself follows the pattern in the example except in three places, the Version and xmlns attributes are swapped round, there is no space in the SIF_Ping (ie ) and there is an additional

 

Any clues would be appreciated.

 

I'm not using any security at present, there is no data being accessed. The OpenZIS server is set to 1.2.

 

Regards

 

@bantonia

 

If you are looking at creating transport layer messages have a look at

 

https://launchpad.net/open-adk-java

 

This is an open source develop kit to speed up agent production.

[bantonia]

I like to develop my own libraries, provides a better understanding of the interfaces in use in the long term. Glutton for punishment! I did download all the files last night for open-adk-java but have not looked at them yet. Is there not a zip file? However, I was able to generate container classes directly from the 1.2 schema using JAXB in NetBeans, fairly painless in the end.

 

Anyway, got a working ZIS now and a SIF_Ping Agent. As far as I can tell my download of OpenZIS does not force authentication, the minimum check is comparing SIF_SourceId. Most of my earlier issues stem from the Apache2 vhosts configuration and not knowing the full URL of the ZIS for the SIF_message to be sent to, very frustrating!

[penfold_99]

I like to develop my own libraries, provides a better understanding of the interfaces in use in the long term. Glutton for punishment! I did download all the files last night for open-adk-java but have not looked at them yet. Is there not a zip file? However, I was able to generate container classes directly from the 1.2 schema using JAXB in NetBeans, fairly painless in the end.

 

Anyway, got a working ZIS now and a SIF_Ping Agent. As far as I can tell my download of OpenZIS does not force authentication, the minimum check is comparing SIF_SourceId. Most of my earlier issues stem from the Apache2 vhosts configuration and not knowing the full URL of the ZIS for the SIF_message to be sent to, very frustrating!

 

I do have the files available in a zip. It would be a single jar and the example files.

 

You are a glutton for punishment if you want to write your own libraries to handle SIF messages. I would strongly recommend the adk as the complexity of SIF at the transport layer is going to slow the process down.

 

We are looking at making a zones available for developers that will have access to different MIS. Will update when I have more information.

[bantonia]

Thanks for the advice, as I say, I am a glutton for punishment so I do agree with you there. My point is that I wish to use GlassFish in the end to support my Agents as servlets with timing events via JMS to each of the servlets to trigger SIF messaging. I realize there will be methods to process the SIF messages within the library but I do not want to by-pass the doPost() entry points etc created by GlassFish by using a separate library, which I expect are also in there. Also there is a certain amount of satisfaction once something challenging comes together! Edited May 1, 2012 by bantonia