Application Development and Testing in today’s interconnected world requires integration and interdependency between the applications and current and/or legacy systems.

To date Service Virtualization has been the preserve of the testing community, using mocked or record/replay technology to provide limited integration testing of applications, prior to implementation in production. The challenge with the current mocked or record/replay technology is that it is static (the recordings represent a series of simplex interactions, that when a change occurs need re-recording).

These static recordings do not simulate the complex end to end integration between applications, nor do they represent the dynamic nature of today’s integrated applications. To use these static recordings to represent a complex interconnection or automated process, across multiple systems requires hard coding and or complex configuration between the recordings, thereby creating another largely static environment, which is difficult to change. This static testing is the basis of the Service Virtualization technology provided by the current leaders in this field. Their customer’s find these static Service Virtualization technologies expensive to develop and maintain.

It is also means they are unable to ‘shift left’ to meet the expectation of agile development and their customers are also not able to meet the demands of the business for fast delivery of new applications/product to market.

Developers need a dynamic Service Virtualization environment, allowing them to use Dev/Ops agile development and continuous integration techniques to improve and integrate their applications.

Ostia Portus EVS technology delivers the dynamic Service Virtualization technology demanded by our customer’s Development and Testing teams. EVS technology builds on established record/replay technology. It extends this capability/function and is able to simulate the request/response interaction, providing a fully customizable approach to creating virtual services. Ostia Portus EVS technology:

· Uses metadata definitions and structures to create services, as a series of components, to simulate the real services, within a Java development environment.

· Uses the simulated service components and Java to integrate and build contextual services to represent the interdependency between services in an end to end environment, where the dynamic response from one service can easily be used within the request in subsequent services. This is critical in representing a complex interconnection or automated process.

The application under test thinks it’s talking to the real end to end system but is talking to simulated service(s) using the Portus EVS framework technology running on commodity hardware and software, which can be cloned as often as necessary, so test systems of this nature can be made available on demand.

The implementation of the contextual service (made up from ‘inter related’ EVS service components) is fully modifiable so that organizations can extensively customize the virtual service implementation to fully represent their end to end application or environment, where multiple different systems are involved.

The user can create a library of EVS service components (for example a Bank’s Payment/Settlement Service, Mobile Banking environment or and external services such as ‘nets’ or SWIFT), simulating each service, or a series of interrelated services. Through the EVS interface, Developers can use the EVS service components, to develop and interactively test/prove their developments to resolve issues within Development, so that final testing is more focused on validation. Testing is completed with the Developers proving their applications as they code, using the simulated end to end services.

Thereby reducing the problems with errors in Test and more importantly in Production, improving the coverage and complexity of testing, and improving overall code quality assurance. The EVS framework allows users to test edge cases like downstream failure that can be difficult to provide in a live environment, and the library of EVS service components will improve regression testing.

1.1.2 The requirement for service virtualization

Most existing and new applications being developed require links to back office services for testing. With the advent of continuous integration and agile development, the availability of such services 24/7 is now becoming a requirement, however, this is rarely possible due to the contention for access to these test systems. Often teams are limited to 3, 2 or even 1 day slots every month to test while it can also take a significant number of days to deliver the test environment. If they cannot complete their testing during that time, they must wait which causes incredible delays in releasing new or modified applications.

In other cases, test systems or applications simply are not available in the environment where testing may take place. This can occur for a number of reasons:

· Lack of access to test systems and data due to data governance restrictions (e.g. offshore development).

· No connectivity to the testing environments which can occur for offshore projects or even for on shore outsourced projects.

· Limited availability of testable systems with accurate, but anonymised data

· In agile development environments, the dependant applications required to complete testing may not exist when required or are unstable as they are being developed by another agile team.

1.1.3 Addressing the requirements

Developers and test teams need a system that can provide the precise response they need immediately, demonstrating some service side behaviour. To create an simulation of the services, a form of request/response semantic, between the application under development and other new applications and or to back office legacy applications. Some examples of the transports and protocols involved are:

· WebSphere MQ services.

· Web services.

· REST services.

· JMS services.

· Simple sockets - TCP/IP services.

· APPC services.

· And there are others.

While these transports provide the means to deliver and receive payload, the payload also may be in different formats such as:

· XML.

· JSON.

· SOAP.

· Flat records.

· EDIFACT.

· SWIFT.

· with the flexibility to add new formats.

1.1.4 Solving the problem with virtual services

Portus EVS solves the problem by simulating the request/response interaction so that the system under test thinks it’s talking to the real test system but is talking to Portus EVS running on commodity hardware and software. As this is a simulated environment it can be cloned as often as necessary, test systems of this nature can be made available on demand.

The traditional way to achieve this has been to record traffic between a system under test and the real test system. This can then be played back to the system under test which believes it is talking to the real test system. This is very useful for regression testing, however, what about the following:

· What if a request has been received that hasn’t been recorded?

· What if the real service is not available to make the recording?

· Running in the Cloud, very few organizations are likely to allow connectivity to their core test systems for recording.

· What if specific and particularly custom logic is required?

· What about data? Data governance regulation is pushing organizations more and more towards testing with synthetic data i.e. valid while not representing data for any person or organization.

· What if the service simply does not exist yet?

· What if the service is continually adapting?

Portus EVS addresses all of these requirements by supporting both the traditional record and replay implementation, through the use of the Portus EVS server implementation, while answering all of the above questions by extending the Portus EVS Server environment with the Portus EVS framework to offer unparalleled flexibility in the creation of virtual services.

1.1.5 Portus EVS Server environment

This focusses on a simple, configuration only approach to allowing the creation of virtual services. It offers the following capabilities:

· An ability to create proxy MQ and Web services that sit between the system under test and the real test system.

· Payloads are configured based on COBOL structures, WSDL definitions or XSD definitions for XML.

· This facilitates the recording of requests and responses as they flow to and from the real test service.

· These recordings may then be replayed to a system under test without the need for the real system to be available.

· This offers further capabilities as follows:

o An ability to create requests and responses based on the recorded ones thus increasing the coverage without having to record each and every request/response pair.

o An ability to mask data which can be used for testing against real services in flight or for the recording of masked datasets for use safely outside of the organization or even outside of the country.

o An ability to modify responses on the fly with external customizable routines written in PHP.

o An ability to use Portus EVS data generation capability for masking or synthetic data generation.

o Integration with common 3^rd party Test Data Management toolsets.

· Finally, this technology also offers a capability to access legacy or hard to reach data sources with view to accessing and masking that data on the fly for safe use during testing.

The skills required to use this are:

· An understanding of the service transports, protocols and payloads being virtualized.

· Reasonable IT knowledge to use the GUI (training available).

The documentation for Portus EVS Server environment can be found here.

1.1.6 The Portus EVS Framework environment

This framework builds on the Portus EVS Server environment by offering a more customizable approach to creating virtual services. Its primary capabilities are as follows:

· Ability to create services using metadata definitions and structure without a requirement to access the real service.

· Service implementation is fully modifiable so that organizations can extensively customize the virtual service implementation to better represent their application or environment.

· Provides support for the virtualization of services not suitable for the recording.

· Enables contextual services to be created such that the effects of calling one service are seen in calls to subsequent services. This is critical for the simulation of a full application thus bringing the testing capability to a new level.

· Can also facilitate end to end testing where multiple different systems are involved.

· Leverages the power of the newer toolsets and technologies developers are using today.

· Fits in perfectly with any Java development methodology currently in use within an organization.

· Ideal for Cloud deployment of test environments as:

o No connectivity is required to the real service.

o All data used can be synthetic or masked thus avoiding the risk of data leaks.

· Easily enables the creation of services that don’t currently exist.

· Can be used for support to simulate user problems.

· Can be used for training as training simply requires a simulate environment.

The skills required to use this are:

· An understanding of the service transports, protocols and payloads being virtualized.

· Reasonable IT knowledge to use the GUI.

· Java programming knowledge.

· Maven knowledge.

For more information, please see Portus EVS Framework.

Back to Contents

1.2 Portus EVS message and data generation

The Portus EVS platform contains a rich set of functionality to create synthetic data of different sorts. These can also be extended based on customer requirements or can be customized such that the customer creates their own data generation routines for specific purposes. The requirement to generate messages and data has always been around, however, with new stricter data governance regulation, there is an increasing need to be able to create schema compliant messages and data that is valid but represents no person or organization for test purposes. This includes (but is not limited) to the following:

· Creation of messages to test batch systems which are driven based on messages.

· Creation of database tables with test data for performance and scalability testing.

· Creation of rich sets of messages to test the edge conditions within applications.

· Creation of language specific messages and data to test the internationalization capability of an application.

· And there are many others.

Once available in the Portus EVS platform, the generated messages and data can be used as part of the service virtualization framework or to generate synthetic data or messages.

1.2.1 Addressing the requirements

The Portus EVS framework is building on the methodology described in the following picture:

The process is as follows:

1. Portus reads metadata which describes the schema or data model to be used to create the messages or data.

2. The user configures data generation routines to be used to create data for each field or node in the metadata.

· The user decides how many sets of the data to publish and the data is then published to the target.

1.2.2 Processing the metadata

The goal is to support any set of inputs that could describe a schema, message or structured set of data can be read by the Portus tool. The following are currently in scope but others may be added in the future:

· XML Schemas (i.e. XSD files).

· COBOL Structures.

· XML Messages.

· JSON Messages.

· ODBC Database schemas.

· SWIFT Messages.

· EDIFACT Messages.

· CSV Messages.

· etc.

Portus will read the metadata and present each field in the metadata to the user defaulted to a fixed value based on the type of the field or node. Portus will also maintain relationship information that is found within the metadata such as referential integrity between database tables or relationships between XML nodes in an XML document.

1.2.3 Selecting the data generation function

For each field in the metadata, the user can then select which data generation routine is to be used to create data for the field when a message or set of data is being generated. This data generation routine will be called for each entity that is to be created based on the metadata.

Once configured, this configuration will be saved by the tool so that it can be reused or modified in the future.

1.2.4 Generating the data

The user then selects how many messages or sets of data that they wish to generate. For example:

· For XML, it is the number of XML messages of the type defined by the schema to generate.

· For JSON, it is the number of JSON messages of the type defined by the sample JSON message to generate.

· For a relational schema, it would be the number of sets of records for that schema to be generated.

· For SWIFT, it will be the number of SWIFT messages to generate.

· etc.

There are a number of targets to which this data can be written:

· Messages or flat records may be written to the file system.

· Data may be written to a relational database such as Oracle, DB2 etc.

· Data may be written to non-relational databases such as VSAM or ADABAS.

· Data may be written to a Portus Web Service wrapping some business logic written in Natural or COBOL.

· Data may be written to an MQ Queue to test that application.

1.2.5 Updating the metadata

Often the metadata for which data is being generated may have to change which is often the case as systems are being developed or improved. The following will be possible to a given base set of metadata:

· Fields may be added to the metadata.

· Fields may be removed from the metadata.

· Fields may be modified in the metadata.

· The data generation routine may be changed for a field in the metadata.

Depending on the target for the data, the following will occur:

· For messages (e.g. XML, JSON, SWIFT, EDIFACT etc.), a new set of messages will always be created.

· For ODBC compliant databases:

o Fields that are added to the metadata will have new data generated into those fields in the database.

o Fields that are modified in the metadata will have the updated value generated into those fields.

o Fields that are deleted in the metadata will be ignored but will remain on the target database unless created again from scratch.

· For non-ODBC compliant databases, the dataset must be created from scratch.

Back to Contents

2 Portus EVS Framework

The framework is focussed on performing repeatable things well and quickly so that that the customer can focus on their actual requirement. Hence the focus is to create a virtual service that simulates how the actual service functions without the extensive complications that the real service must deal with. The framework also offers a level of control and customization that facilitates full integration with continuous integration and testing environments.

The framework consists of the following:

· A number of wizards to guide you through the creation of each type of virtual service.

· A run time environment that does the heavy lifting around transports, protocols and payload support.

· A run time environment that offers additional helper functions particularly in the area of data generation capability.

· A run time environment that is configurable on the fly and thus capable of changing based on environment conditions or dynamically based on the requirements of the test being run.

· An initial implementation of the virtual service in Java that can be adapted and customised based on the user requirements.

The creation of virtual services and their specific requirements and configuration is described in detail in various tutorials here. The following is a description of the standard elements of all virtual services created by the Portus EVS Framework.

Back to Contents

2.1 The capabilities of a Portus EVS virtual service

There are a number of ways in which a Portus EVS virtual service may be used:

· In the simplest case and in the basic service that is created initially, the service will accept a request and return a default response depending on the metadata available.

· The virtual service may also be configured to call the real service and thus return the real response to the caller. In this mode, if the call to the real service fails, the user may configure whether the service should proceed and return a virtual response either from replay or by calling the virtual service implementation.

· Recording may be activated which will record the response returned by a service call based on keys provided by the user. These keys will indicate what values in the request should be used to identify the request uniquely. These keys are then used to create a file name to which the payload and potentially transport or protocol specific data to a recordings directory. Note that when recording is active, the response from the real, virtual or a replayed service request will be saved. When a recording already exists for a specific key, it will be overwritten.

· The service may run in replay mode. Replay mode uses the keys from the request to understand the unique filename used to record the response and determine if a response already exists. If it does, the recorded response is returned to the caller.

· In the case of a response from the virtual service implementation or from a recording, it’s possible to set a minimum and maximum delay in milliseconds. Portus will wait for a random amount of time between these values before responding to the caller thus more accurately mimicking the real service. This may also be used for non-functional testing by forcing the service to respond in a time that is out of the SLA to test how clients will react.

There are other potential uses for these services:

· The services may be used as a cache for a service. If a given service is likely to return the same result for a given request over a period of time, this response could be recorded so that from that moment on, the replay mechanism will be used to respond to the service request. The recording could then potentially be deleted after a specific period of time such that the real service is called and thus the response updated after an appropriate interval.

· If real time modification of payloads is required (e.g. for data masking) it would be possible to call a virtual service helper to allow the modification of the response form the real service.

Back to Contents

2.2 The Virtual Service Project

When a virtual service is created, Portus creates a complete Apache Maven (build automation tool) project. That Maven project may then be imported into the Integrated Development Environment (IDE) of your choice and may then be managed, modified and deployed from there. This offers significant advantages:

· As a Maven project, it will be familiar to Java developers who use Maven extensively for development.

· It can benefit from all of the editing, helper and other functions offered by the IDE.

· It can be committed to a source control system and thus managed in exactly the same way as any other Java project.

· It can be run using the debugging capability in your IDE.

· It can use existing Java functionality available within your organization.

· Any dependencies between virtual service projects may be managed in this way also.

As the final package is an Application Server WAR file, it must then be deployed to a clone environment for usage when it has been tested locally. It can also potentially be deployed to application servers in various PaaS environments such as IBM’s Bluemix environment.

Back to Contents

2.3 The Portus EVS Project Created by Sandbox Generation

When a Portus EVS project is created using the Portus tools, a standard structure is created with some limited additions for different types of projects.

2.3.1 The Base Project

The base project is a standard Java project, when generated and imported into Eclipse, it will look like the following:

The pom.xml file in the base directory is the standard pom file required for maven projects. Each of the directories and its contents are described in the next sections.

Note the term “<groupid>” in the following must be replaced with the Maven group id you use in your maven projects.

2.3.1.1 /src/main/java

This section will contain the packages and code created by EVS generation and can potentially be added to as the project develops:

- Package “<groupid>.generated.sv.impl” contains the java code for the project.

o VirtualServiceImpl.java (ServiceImp.java in newer projects) will be generated the first time and is the logic that will be called by the framework to create the rules for the sandbox. It is intended that this will be modified as required by the user and thus when it already exists, it will not be overwritten.

o VirtualServiceImplGenerated.java (ServiceImplGenerated.java in newer projects) will always be generated if VirtualServiceImpl.java (ServiceImp.java in newer projects) exists. This will enable you to make changes to the project as it develops and generate the base code to reflect those changes. This ‘base code’ can then be added to your real implementation and modified as appropriate.

- Package “<groupid>. servlet” contains servlet code required by the project.

o VirtualServiceServlet.java is required to get control to the appropriate point in the EVS framework and must not be modified.

It is expected that as a project develops, further code and packages will be added to this directory.

For sockets projects, the following package will also exist:

- Package “<groupid>.impl” contains helper code for the sockets project.

o VirtualServiceHelper.java is provided as part of the sockets sandbox support. If the payloads that will be received over sockets are variable, this must be modified by the developer to correctly identify to the EVS framework what length it should expect for the incoming message. Please refer to the “Portus EVS sockets transport” documentation for more information.

2.3.1.2 /src/main/resources

This section will contain resources used by the sandbox implementation:

- Directory “payloads”

o This directory will contain all of the payload meta data required for the project.

- File “payloads.properties” contains the control statements for the processing of the payloads in the payloads directory. This should only be modified if requested by Ostia support.

- File “logback.xml” contains the logging statements for the project and can be modified to assist with debugging a project.

- File “<project name>.properties”, where “<project name>” is the name of the project, contains the EVS framework properties for this project. This should only be modified if requested by Ostia support.

2.3.1.3 /src/main/webapp

This section will contain resources used by the sandbox implementation as part of the web application generation:

- File “index.html” will contain a HTML page which will be presented if the service is accessed without parameters.

- Directory “WEB-INF” will contain the web.xml file that is used to control behaviour when deployed in the application server or during testing under jetty. This must not be modified unless instructed to do so by Ostia support.

- For web services projects only, this will contain the WSDL and any imported XSDs required for the service based on the definition when the service was created.

2.3.1.4 /src/test/java

This section will contain java packages and code for testing:

- Package “<groupid>.test” contains test java code for the project.

o UnitTest.java will always be generated into the project. This is a simple test case that ensures that the service will start correctly. It does not test any of the functionality.

o TestVirtualServiceImplementation.java will only exist if the project is built with the property generateUnittest=true. Please refer to the section on comprehensive unit test generation for more details.

It is expected that as a project develops, further code and packages will be added to this directory.

2.3.2 The Standard Generated Implementation

The VirtualServiceImpl.java (ServiceImp.java in newer projects) will contain the initial skeleton code for the sandbox you wish to create. For each function required, the following will be generated:

- A Pojo (plain old java object) based on the input data format will be passed to the function.

- The code will print out each element received as part of the input message. This will provide a helpful sample of how to access the incoming elements.

- The code will then create a response Pojo based on the response data format. This will be filled out with random generated data.

- This is returned to the framework.

Note that for MQ, JMS and Sockets protocols, there will only be one function. For REST and SOAP protocols, one or more functions will be created depending on how many REST or SOAP methods the service must support.

2.3.3 Generating a Comprehensive Unit test

The framework will also optionally generate a comprehensive unit test called TestVirtualServiceImpl.ava (TestServiceImpl.java in newer projects). This is triggered by specifying the following property on the maven build:

-DgenerateUnittest=true

Note that if this is specified and the TestVirtualServiceImpl.ava (TestServiceImpl.java in newer projects) file already exists, it will _not_ be overwritten as it is likely to contain user changes.

The resulting generated code will contain code to start a version of the virtual service using jetty for test purposes, run a number of unit tests and subsequently bring the jetty service down.

In addition, this will contain the initial skeleton code to drive each function in the virtual service. For each function in the service, the following will be generated:

- A Pojo (plain old java object) based on the request data format for the function will be created.

- This will be sent to the virtual service using the appropriate protocol.

- The response will be received from the virtual service using the appropriate protocol.

- The response Pojo will be checked to ensure that each field in the response has a value (as will be filled out by the standard virtual service implementation) and if not, an assert will be triggered.

The purpose for generating this comprehensive test is two folded:

1. It is intended that this can be extended to create real test cases to drive and test the real service and understand what is valid or invalid behaviour.

a. This can be used for real testing of the actual service.

b. It can also be run regularly to check for changed behaviour in the actual service which should trigger an update to the virtual service implementation. Of course, this should be flagged in advance between teams but this represents a non-human ‘sanity check’.

2. This can then be use to drive the virtual service to ensure that the virtual service is performing correctly.

a. This will ensure that when the service is built, the build will only succeed if the virtual service is performing correctly.

The default behaviour is not to generate this unit test as with any existing projects, the defaults are likely to cause the test to fail as changes will have been made in existing projects. You may force the generation of this for existing projects by specifying the option on the build.

The intention is that for newly generated projects (with the exception of Web Services project) from the GUI, this will be generated by default when the initial project is built.

Web Services projects are slightly more complex as there is already a facility to create a standard XML response for each function using configured values. In order to force the generation of the standard Virtual Service Implementation described above for a web services project, add the following option on the maven build for the project:

-DgenerateXMLResponses=false

This will also force the TestVirtualServiceImpl.ava (TestServiceImpl.java in newer projects) to be created. If you simply wish to generate the TestVirtualServiceImpl.ava (TestServiceImpl.java in newer projects), use the standard approach:

-DgenerateUnittest=true

Back to Contents

2.4 Project Components

The EVS framework allows developers to create sandboxes quickly and easily in a few simple steps. This produces a standardised virtual service sandbox which simulates the real service based on the metadata and payloads provided. The result is a Java Maven project which can be imported into any Java IDE and expanded with additional logic as required.

EVS uses standard Java and Maven tools, and so anyone with knowledge in these areas will be confrontable working with the tool and resulting projects. In this chapter, we will cover EVS Sandbox project structure and components.

Throughout this chapter we will be using MQ-COBOL-TEST sandbox as our project example. The structure of an EVS project will be the same or similar no matter what transport or payloads are used. Differences will be highlighted when relevant.

2.4.1 Project Layout Overview:

• src/main/java contains java packages

• target/generated-sources/payloads
contains generated packages

• src/test/java contains unit tests

• src/main/resources contains project resources

2.4.2 Project Payloads (Except WSDL Projects)

During project creation, the Manage Project GUI copies all payloads and their dependencies to the src/main/resources/payloads/ directory prior to the project build process.

Internal QA process to copy all payloads and their dependencies there during automated QA runs.

These payloads are referenced during the build via payloads.properties for most projects, though COBOL payloads are referenced during runtime.

2.4.3 payload.properties – location and use

payload.properties is located in src/main/resources/. Its function is to Identify and categorize all payloads used in the project. This properties file contains two main properties:

payloadBuild.n - where ‘n’ is a sequential number from 0 to the number of payloads to be defined. Note if a number is skipped, higher numbers will not be processed.

payloadGenerateVersion=v - where ‘v’ is ‘0’ (Legacy) or ‘1’ (current)

E.g. for a COBOL project using Request.cpy and Response.cpy you will find entries such as:

payloadBuild.1=Request,COBOL,Request.cpy?<additional params>
payloadBuild.0=Response,COBOL,Response.cpy?<additional params>
payloadGenerateVersion=1

This properties file should not be modified manually unless requested to do so by Ostia support.

2.4.4 Overview of payload.properties structure:

· payloadBuild.n=<payloadid>,<payloadformat>,<payloadfilename>

o <payloadid> unique name by which this payload is known.

§ For XSD format, this should be the name of the complex element in the payload file that this payload represents (where there are multiple within the XSD file)

· <payloadformat> the format of the payload:

o RAW: no format and no <payloadfilename> may be provided.

o XSD: <payloadfilename> contains an XML Schema.

o JSON: <payloadfilename> contains a sample json message

o JSONSCHEMA: <payloadfilename> contains a sample json schema

o COBOL: <payloadfilename> contains a COBOL structure definition

· <payloadfilename> is the name of the file in the payloads directory.

o For COBOL payloads, this also contains further processing instructions for COBOL such as the COBOL dialect, the size of the lines used and other options to deal with COBOL structures from different compilers.

2.4.5 payloads.properties Example

#Payload Properties

#Thu May 10 17:40:13 BST 2018

payloadBuild.6=PutRequestJSON,JSON,put_req.json

payloadBuild.5=PutResponseJSON,JSON,put_req.json

payloadBuild.4=RawData,RAW

payloadBuild.3=GetWeatherResponse,XSD,GetWeatherResponse.xsd

payloadBuild.2=GetWeatherRequest,XSD,GetWeatherRequest.xsd

payloadBuild.1=CPL002,COBOL,Request.cpy?codepage\=UTF-8&dialect\=FMT_INTEL&columns\=USE_LONG_LINE&org\=IO_FIXED_LENGTH&split\=SPLIT_NONE

payloadBuild.0=CPL004,COBOL,Response.cpy?codepage\=UTF-8&dialect\=FMT_INTEL&columns\=USE_LONG_LINE&org\=IO_FIXED_LENGTH&split\=SPLIT_NONE

payloadGenerateVersion=1

2.4.6 Overview of the properties for the project:

2.4.7 <project name>. properties – location and use

<project name>.properties is located in /src/main/resources/. The function of this file is to provide EVS project specific properties for the project.

Contents:

• Transport related properties (MQ queue names, JMS details etc.)

• Generic EVS behavioural properties

• Function properties depending on the property type

This properties file should not be modified manually unless requested to do so by Ostia support.

2.4.8 <project name>.properties – MQ/JMS Functions

• function.n where ‘n’ is a sequential number from 0 to the number of functions to be defined. Note if a number is skipped, higher numbers will not be processed.

• function.n=<functionid>,<functionimpl>,<requestpid>,<responsepid>

• <functionid> unique name by which this function is known.

• <functionimpl> the name of the method in the virtualServiceImpl.java

• <requestpid> the payload id of the request to be passed to the method

• <responsepid> the payload id of the response to be returned from the method

• Note, the payload ids of the payloads referenced above related to those defined in the payload.properties file.

2.4.9 <project name>.properties – MQ Example

#Thu May 10 17:39:59 BST 2018

mqPassword=

mqUserid=

mqServiceQManager=MQ.PORTUS

mqServicePort=1414

mqServiceInputQueue=JPO.SERVICE.INPUT

mqServerConn=OSTIA.SVRCONN

mqServiceOutputQueue=JPO.SERVICE.OUTPUT

mqServiceHost=lxserver.ost.local

mqCopyMsgidToCorrelationId=No

Function.0=CobolFuncID,CobolFuncImpl,CPL002,CPL004

Function.1=XMLFuncID,XMLFuncImpl,XmlRequest,XmlResponse

mqQManager=MQ.PORTUS

mqServicePassword=

mqServiceUserid=

mqInputQueue=JPO.PROXY.INPUT

mqPort=1414

mqHost=lxserver.ost.local

mqOutputQueue=JPO.PROXY.OUTPUT

mqServiceServerConn=OSTIA.SVRCONN

2.4.10 <project name>.properties – REST Functions

• RestFunction.n where ‘n’ is a sequential number from 0 to the number of functions to be defined. Note if a number is skipped, higher numbers will not be processed.

• RestFunction.n=<REST method>,<path>,<impl>,<requestpid>,<responsepid>

– <REST Method> Method (ie GET, PUT, POST etc.) this represents

– <path> JAVA Pattern to match the URL for this request

– <impl> the name of the method in the virtualServiceImpl.java

– <requestpid> the payload id of the request to be passed to the method for POST, PUT and PATCH

– <responsepid> the payload id of the response to be returned from the method for all types except HEAD

2.4.11 <project name>.properties - REST Example

#Fri Apr 20 17:24:35 BST 2018

serviceHost=localhost

RestFunction.9=POST,^/payments/retail/domestic$,paymentsretaildomesticPOST,PaymentsretaildomesticPOSTRequest,PaymentsretaildomesticPOSTResponse201

RestFunction.8=PATCH,^/customers/individual$,customersindividualPATCH,CustomersindividualPATCHRequest,CustomersindividualPATCHResponse200

RestFunction.7=POST,^/transfers/retail/domestic$,transfersretaildomesticPOST,TransfersretaildomesticPOSTRequest,TransfersretaildomesticPOSTResponse201

RestFunction.6=PATCH,^/cards/[^/]+/[^/]+/[^/]+/actions/block$,cards_cardId__cardSequence__primaryCard_actionsblockPATCH,Cards_cardId__cardSequence__primaryCard_actionsblockPATCHRequest,Cards_cardId__cardSequence__primaryCard_actionsblockPATCHResponse200

RestFunction.5=GET,^/accounts/[^/]+/transactions$,accounts_accountNumber_transactionsGET,Accounts_accountNumber_transactionsGETResponse200

RestFunction.4=GET,^/accounts/[^/]+/balances$,accounts_accountNumber_balancesGET,Accounts_accountNumber_balancesGETResponse200

RestFunction.3=GET,^/accounts/balances$,accountsbalancesGET,AccountsbalancesGETResponse200

RestFunction.2=PATCH,^/cards/[^/]+/[^/]+/[^/]+/actions/unblock$,cards_cardId__cardSequence__primaryCard_actionsunblockPATCH,Cards_cardId__cardSequence__primaryCard_actionsunblockPATCHRequest,Cards_cardId__cardSequence__primaryCard_actionsunblockPATCHResponse200

RestFunction.1=GET,^/customers/individual$,customersindividualGET,CustomersindividualGETResponse200

RestFunction.0=POST,^/customers/individual$,customersindividualPOST,CustomersindividualPOSTRequest,CustomersindividualPOSTResponse201

restSwagger=openlayerswagger.json

servicePort=8080

2.4.12 <project name>.properties – Sockets Functions

• requestPayload is the payload id of the data to be passed to the implementation method

• responsePayload is the payload id of the data to be returned from the implementation method.

• proxyPort is the port upon which the implementation will wait to accept requests or to receive an incoming message depending on the configuration.

• requestLength is the default expected request length

• responseLength is the default expected response length from the real service when called.

• socketsInitiateSocket determines how the connection is initiated

– Connect will cause the implementation to connect to a socket and wait on a receive.

– Accept will cause the implementation to issue an accept and wait on connects before issuing a receive.

• socketsReceiveTimeout the time a receive will wait for incoming data. If this is specified as ‘0’ it will never time out.

2.4.13 <project name>.properties – Sockets Example

#Wed Jul 26 19:49:04 BST 2017

responsePayload=NETS_OUT_HDR

servicePort=2221

requestPayload=NETS_IN_HDR

serviceHost=localhost

proxyPort=52000

requestLength=632

responseLength=902

socketsInitiateSocket=Connect

socketsReceiveTimeout=0

2.4.14 <project name>.properties - WSDL Example

In the wsdl properties below wsdl = the name of the WSDL which was used to build the project

Example

#Tue Nov 07 18:06:39 GMT 2017
wsdl=CardApplicationService.svc.wsdl

2.4.15 Further details about files and directories within the project:

2.4.16 <project name>_mapping.xml

• Located in src/main/resources/

• Provides a mapping between fields in the implementation and services and fields in the data model

• Contains an entry for every class and field referenced

• If it already exists, a new xml will be saved as <project name>_mappingGenerated.xml

• Should not be modified manually unless requested to do so by Ostia support.

2.4.17 <service name>_1_0_mapping.xml

• Located in src/main/resources/

• One created for each service created as part of the data model

• Provides a mapping for each of the Portus Integrate services created during the build process

• Will be overwritten each time a build is completed

• Should not be modified manually unless requested to do so by Ostia support.

2.4.18 Java Code – src/main/java/

• Package <groupid>.generated.sv.impl

– <groupid> is the groupid used on the build

– Contains VirtualServiceImpl.java (ServiceImp.java in newer projects) which it is intended will be modified.

– VirtualServiceImplGenerated.java (ServiceImplGenerated.java in newer projects) will be generated when VirtualServiceImpl.java (ServiceImp.java in newer projects) already exists as a reference.

• Package <groupid>.servlet

– Contains VirtualServiceServlet.java to represent the transport in use

– Will also contain PatchServletClass.java for REST projects

– Should never be modified manually.

• Add helper packages

– Will not be modified by Portus EVS

– Will facilitate easier upgrading when the transports or meta data changes

2.4.19 Java Code – src/test/java/

• Package <groupid>.test

– <groupid> is the groupid used on the build

– Will contain UnitTest.java

• Generated as part of the archtype generate

• Simple unit test to ensure the project has been built correctly and will start up

• May contain TestVirtualServiceImpl.ava (TestServiceImpl.java in newer projects) which is generated when –DgenerateUnitTest=true is specified on the build.

• TestVirtualServiceImpl.ava (TestServiceImpl.java in newer projects) is a comprehensive unit test to drive each method generated as part of VirtualServiceImpl.java (ServiceImp.java in newer projects) testing the appropriate request and response payloads.

– It is intended it will be modified and/or used as a base for other tests and will not be overwritten once it already exists, instead TestVirtualServiceImplGenerated.java will be generated whenTestVirtualServiceImpl.ava ( TestServiceImpl.java in newer projects) already exists.

• Add helper packages

– Will not be modified by Portus EVS

– Will facilitate easier upgrading as new versions are created

2.4.20 Generated Java Sources

• Located in target/generated/payloads/

• One or more created for each class created as a result of the JSON or XSD mapping

• Used to map JSON/XML to POJOs and back again as part of the framework process

• Will be overwritten each time a build is completed

• Should never be modified manually.

2.4.21 Portus EVS Data Service Helper Classes

• Located in target/generated/payloads/portusCrudCode

• One generated for each service defined in the data model

• Provide capability to add, delete, update, list or get for each service mapping from or to the appropriate Java classes

• Will be overwritten each time a build is completed

• Should never be modified manually.

2.4.22 Debugging – logback.xml

• Located in src/main/resources

• Trace Portus EVS framework code with the following entry:

<logger name=“com.ostiasolutions" level=“DEBUG" additivity="false">

<appender-ref ref="STDOUT" />

</logger>

• Trace user code with the following entry:

<logger name=“<groupdid>" level=“DEBUG" additivity="false">

<appender-ref ref="STDOUT" />

</logger>

– Where ‘<groupid>’ is the group id used to create the project

2.4.23 Logback.xml - Example

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE xml>

<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>

</encoder>

</appender>

<appender-ref ref="STDOUT" />

</logger>

<appender-ref ref="STDOUT" />

</logger>

<appender-ref ref="STDOUT" />

</logger>

<appender-ref ref="STDOUT" />

</root>

</configuration>

2.4.24 Building a Project using the GUI – Part 1

• Collect project name and location

• Collect transport properties (e.g. MQ Queues, JMS Queues etc.)

• Collect payloads

– IDs and payload types (e.g. XSD, JSON etc.)

– Load contents (including all dependencies) into GUI

• Collect functions (For SWAGGER/WSDL this is automatic)

– Map functions to implementation names

– Identify request and response payloads (where appropriate) for each function

• Proceed to project creation

2.4.25 Building a Project using the GUI – Part 2

• Create the project using archetype generate

• Write payloads to /src/main/resources/payloads/ in the new project

• Write payloads.properties to /src/main/resources/ in the new project

• Write <project name>.properties to /src/main/resources/ in the new project

• Invoke the mojo for the appropriate transport

2.4.26 Building a Project using the GUI – Part 3

• Process the payloads

– Create classes using JAXB for XSDs

– Create classes using JSON2POJO for JSON files or JSONSCHEMAS

– No pre processing is required for COBOL structures

• Compile all the created JAVA Classes

– This is required so that the compiled classes are available

• Create VirtualServiceImpl.java (ServiceImp.java in newer projects) in src/main/java/ based on the methods defined and the request and response payloads.

• Optionally create TestVirtualServiceImpl.ava ( TestServiceImpl.java in newer projects) in src/test/java/ based on the methods defined and the request and response payloads.

• Optionally process the data model

– Create the data services and tables based on the payloads.

– Write one record to each service as a base

– Create a CRUD helper java class for each service

2.4.27 Building a Project using command line mojo

• Create the project using archetype generate

• Process the payloads

– If payloadsparms is specified, the payloads are defined with an absolute or relative file location. This causes the payloads to be written to to /src/main/resources/payloads/ in the new project and the payloads properties to be created.

– If individual request or response payloads are provided, these are then mapped to the new payload format and the files written to /src/main/resources/payloads/

– Write the payloads.properties to src/main/resources/

• Process the functions

– If functionparms is specified, the function.n properties are built in the project properties.

– Otherwise, standard request/response project properties are built

– Write the <project name>. properties to src/main/resources/

• The process can now be built in exactly the same way as the third part of the GUI process.

2.5 Project EVS configuration

With all virtual service projects created by Portus, there is fixed configuration that must live for the lifetime of that project and which cannot be changed which includes things like virtual service implementation class names and a unique name per virtual service. These configuration elements are included in the web.xml for the project as ‘init-parm’ values and these must never change.

Note that a key init-parm for Portus is the one named ‘webapp-name’ as shown below:

<init-param>

<param-name>webapp-name</param-name>

<param-value>uniquevirtualservicename</param-value>

</init-param>

This is the name used by Portus when writing configuration information or other data specific to that particular virtual service.

2.6 Service configuration

Each virtual service project created by Portus will also include configuration for the actual service itself which will depend on the transport used and the payloads being supported for that service. For each project a ‘<webapp-name>.properties’ file is created as part of the build and written to the ‘/src/main/resources’ directory in the project so that it is deployed in the package in the classpath.

When the project is first initialized in an application server container, this set of properties will be written to a new directory outside of the project with the same filename. This directory is relative to the default path active when the project is running as follows:

“../conf/portus/”

These service configuration parameters are expected to be modified potentially as the virtual service is developed and thus the configuration parameters must be modified in the “../conf/portus/” directory. The intention here is that when a project is rebuilt or redeployed, it will continue to use these service configuration parameters outside of the project.

If you wish to reset to the parameters generated into the project, simply delete the copy in the “../conf/portus/” directory and restart the application and it will rewrite the project properties to this directory.

The contents of the properties file are different depending on the transport and the payload in use and are thus described in the relevant section of this documentation where the transports and payloads are described.

2.7 Run time configuration

The ‘Run Time’ configuration is the configuration for the project which may change while the project is executing. For example, there may be a requirement to call the real service during certain times but to call the virtual service in other cases. For each project, the run time configuration default will be written to the “../conf/portus/” directory as <webapp-name>.xml.

Note that it is currently hardened to this directly only when the application is terminated gracefully.

The following table describes the run time parameters, their meaning and their potential values:

Configuration Parameter	Values	Description
callRealService	Yes/No	Determines whether the real service will be called or not. Default: No
callVirtualServiceifRealServiceFails	Yes/No	Determines if the processing will continue to look for a recording (subject to configuration) or call the virtual service if the call to the real service fails. Note that ‘failure’ is a relative term as a call to the real service could return some data so from a Portus perspective that is not a failure. A failure is only considered to be when a response is not returned from the real service. Default: No
maxDelay	Number	This is the maximum number of milliseconds that Portus will wait before responding with a recorded response or a response from the virtual service. This must always be equal to or higher than minDelay. Default: 2000
minDelay	Number	This is the minimum number of milliseconds that Portus will wait before responding with a recorded response or a response from the virtual service. This must always be equal to or lower than maxDelay. Default: 500
recording	Yes/No	This dictates if recording is active or not. Note that when recording is active, any response whether from the real service, a previous recording, or from the virtual service will be recorded. Default: No
recordingName	String	This is the low level name of the directory into which recording files for this project will be written. Default: newRecording
recordingsDirectory	String	This is the high level name of the directory, concatenated with recordingName, into which recording files for this project will be written. This is intended to enable recordings from different projects to be written together without clashing due to the potential presence of a different recordingName for each virtual service. Default: recordingsDirectory
replaying	Yes/No	This determines whether Portus will look for a previous recording for a given request. This comes into effect if the real service is not called or, the real service is called, fails and the configuration has callVirtualServiceifRealServiceFails=Yes. Default: No

The run time values may be changed using the monitoring and configuration wizard accessible from the main Power User menu.

As it is exposed for each service as a simple get/set web service, the user can potentially call this web service to change run time parameters as required for automation purposes. Great case must be taken here to ensure that such changes do not result in unforeseen consequences.

2.8 Monitoring Application

In order to monitor projects running on an application server, the monitoring application must be running on that server before connecting via the monitoring wizard. The monitoring application is provided with your installation in the in the Monitoring Application folder located in your installation directory:

To add this to your application server, simply add the monitoring-1.0.war file to the webapps (or equivalent) folder of your servlet. Ensure the application server and monitoring application are up and running before connecting via the Monitoring wizard GUI.

Back to Contents

2.9 Portus EVS Monitoring and Run Time Configuration

The Portus EVS framework offers an ability to monitor projects as they are running and to change certain run time values in the running system. This is achieved using a Monitoring GUI available from the Portus EVS main menu screen.

2.9.1 Entities that can be Monitored

Portus EVS has a hierarchy of entities that can be monitored using the GUI:

- Individual Portus EVS projects. This is a Portus EVS project where stats may be returned and the run time configuration may be modified by the GUI.

- Application Servers where Portus EVS projects are running. The GUI can list each of the Portus EVS projects running within an application server and allow the selection of each to provide details on each EVS project within the application server.

- Dockers running multiple Application Servers with Portus EVS projects running within them. The GUI will list all of the running instances on the Docker instance. It will then facilitate the selection of any application server instance to allow drill down to the Portus EVS projects running within the application servers.

There are a number of pre-requisites which must be in place before this can be achieved:

1. For application servers, the Portus monitoring application must be running within the application server.

2. For Docker, the remote API interface must be enabled. See Official dockerd Documentation

2.9.2 Overall Concept

On the main menu, it’s possible to add any type of entity which can then be accessed directly from the main menu. Any entity that has been added to the main menu, may also be accessed directly using the URL available for the entity when navigated to from the main menu.

When an entity is accessed from the second level from the main menu, this _cannot_ be accessed using the URL for the entity. Consider the following example:

- A Docker instance is defined on the main menu on host <host> and port <port>.

- When selected, this will list all of the Application Server instances running on that Docker instance.

o The URL for this page can be used to directly access that page.

- You may then select an Application Server instance which will display a list of Portus EVS projects running within that Application Server Instance.

o This page may _not_ be accessed directly with the URL for this page.

o You will find a button to enable you to return to the previous Docker display on this page.

- You may further select an individual Portus EVS project from this page to show the configuration and statistics for that particular project.

o This page may _not_ be accessed directly with the URL for this page.

o You will find a button to enable you to return to the previous Application Server display on this page.

o You will also find a button to enable you to return to the previous Docker display on this page.

There are some other considerations here:

- If an Application Server is defined and accessed from the main menu:

o The URL for this page can be used to directly access that page.

o You will only see a button to allow you to return to the main menu when the Application Server is accessed in this way.

o If a Portus EVS project is selected from this page

§ You will see a button enabling you to return to the Application Server page.

- If Portus EVS project is defined and accessed from the main menu:

o The URL for this page can be used to directly access that page.

o You will only see a button to allow you to return to the main menu when the Portus EVS project is accessed in this way.

2.9.3 Adding Entities to the Main Menu

The following sections describe the process of adding projects, servers and Docker instances to monitor via the monitoring tool.

2.9.4 Displaying a Docker Instance

From the main Monitoring menu, select the ‘Add’ button:

Select the type of entity you want to monitor, in this example we will select Docker:

Fill in:

- Hostname – The machine where your Docker is running

- Port – The port on which your Docker remote API is listening.

- Entity ID – A unique name you give to this monitoring entry. (This will default to the host and IP address concatenated with the Portus Project where appropriate.)

Example of filled out Monitoring form:

Hit ‘OK’ and the new entry will appear in the menu list:

Once an entity has been added, you can select it and hit ‘Proceed’ (or double click on the item) to drill down to the entity details. In the following screenshot, we can see this Docker instance runs a number containers hosting various EVS projects. We can see the base image name from which the container was created, the running status, port type and the public port on which the container is listening:

From here, you can select an individual container to see the details, in the following shot we see the demos_mq_cobol container is hosting a single project – MQ-COBOL-VS-1.0.

You can then select this project to view configuration details. The configuration page is covered in more detail later on in this document:

2.9.5 Displaying an Application Server Instance

From the main menu, select the ‘Add’ button and choose ‘App_Server’ entity type from the dropdown list. Fill in the details for your app server, in this case we have a tomcat running on port 8085 of a local machine and have given it the Entity ID of Localhost Tomcat

Hit ‘OK’ and the new App Server will appear in the menu list:

You can then select the app server entry and click ‘Proceed’ (or double click on the entry) to view the projects on the Server:

Select a project to view configuration options:

2.9.6 Displaying a Portus EVS Project Instance

From the main menu, click the ‘Add’ button and enter the required details:

- Select ‘EVS_Project’ as the entity type from the dropdown list

- Enter the hostname or IP for the machine where the project is running

- Enter the port number the project is listening on – in this example our project is running in a local tomcat configured to use port 8085

- Enter the EVS Project name. Depending on your application version, you may need to enter a forward slash before the project name to correctly pick it up.

- Give your project a unique Entity ID

Click OK once these details have been added, and the Project will appear on the menu list:

Select one of the listed Projects and click ‘Proceed’ (or double click on the entry) to view configuration options for that project.

2.9.7 Updating a Portus EVS Project Run Time Configuration

From the Portus EVS project page, you may click on a configuration item and you will be offered the ability to modify the value for that configuration item. This can be done as follows:

Select an Entity and click ‘Proceed’:

Select an Item from the list, in this case, a Docker container, click once to view the projects running in this container:

Select a project from the list, click once to view configuration options:

The configuration options for the selected project are displayed:

Select a configuration to modify and update the values, here we are changing the max delay to 8,000:

Click ‘OK’ to apply the changes, and the new value will now be reflected in the configuration options window alongside the original value:

Once modified, the new value will be shown in the right column. You may select and change multiple configuration items in the same way. If you wish to confirm the updates, hit the ‘update’ button and the configuration will be updated if changes are made. If you wish to remove the proposed configuration changes, simply hit the ‘refresh’ button and the page will revert to the existing project configuration.

At the bottom of the Configuration Options page, users can view the number of requests and executions that have been performed on a selected service:

Back to Contents

2.10 Portus EVS Data Model Creation

The creation of sandboxes and test environments alone is really only part of the story required in a test environment. A critical part of the testing environment is access to a data model that can be easily traversed along with data that can be used for testing. Having isolated test data is never more important than today with the advent of the GDPR regulation in Europe. Portus EVS is perfectly placed to manage this requirement.

2.10.1 Background

In the vast majority of cases, the ‘data model’ as it stands in the existing systems has evolved rather than having been designed from the start. This has resulted in many anomalies in data, a lot of duplication and data structures that are sometimes difficult to understand. In the real test environments, a lot of effort is required to replicate this ‘model’ for good reason; it must correctly represent the production environment.

Portus EVS simulates back office services and therefore accepts and delivers data from this model to the applications under test, however, this does not mean it must replicate this model. Once the data being presented to the applications and the formats are correct, the front end applications don’t need to understand or know about the back office model. In fact it makes no sense to replicate this in a simulated environment.

For this reason, Portus EVS takes a much more pragmatic approach creating a data model based on the payloads in the messages sent to and received from simulated applications. This results in a much cleaner and simpler to understand data model for testing and means that test data can more easily be created from synthetic data sources thus fully complying with GDPR and internal data governance rules and regulations.

2.10.2 How does it work?

Portus EVS is uniquely positioned to create this model when building simulations and sandboxes for organizations. In all cases, Portus EVS is given the Meta data for the messages that are sent to a simulated application and the Meta data received from a simulated application. This Meta data has the following information:

- The individual fields or ‘pieces of data’ that make up the requests and responses.

- The relationships between those fields within each request response.

- The relationship between the fields in the requests and the responses.

- The relationship between request and response fields between different application calls.

This enables Portus EVS to gather related fields together from requests and responses and to use Portus Connect to create services backed by database tables to hold data related to requests and responses. As part of the initial setup, a single record with random data is added for each service. Once created, the Portus Monitoring and Configuration GUI can be used to access the data using those services and to add, update or delete that data as required using a GUI.

For larger amounts of data, Excel spreadsheets may be used to upload data directly to the tables backing up the Portus Connect services.

The standard skeletons for each simulated service is then designed to call into Portus EVS to get at the data required. When Portus Connect has been configured in the run time, the data will be retrieved from the data services created as part of the data model. When not configured, the skeleton will simply return random data as before.

2.10.3 Separation of Sandbox and Data

A key element of the implementation of the data model in Portus EVS is that there is a clear separation of the sandboxes, where the code resides and runs, and the data services. This is intentional as depending on the testing required, there is a requirement to adopt different configurations. For example:

- If the sandbox is being used for intensive development, it is likely that it is safer and better practice for the developer to have a standalone sandbox with a standalone set of data services. Thus the developer can only screw their own environment up and nothing is shared.

- In a CI environment, it is likely that a consistent set of data services with a consistent state is required to avoid CI processes and test falling over due to inconsistent data.

- In an integrated environment, the requirement for stability and related data is more acute.

Portus EVS achieves this goal by enabling the configuration of which Portus Connect server to use for a given instance of a sandbox.

2.10.4 Data Types Used in the Model

The model uses strictly the string data type with a view to ensuring that any type of testing is possible whether the data exists or not, whether it is valid data or not so that any type of negative test may be created using the data.

2.10.5 Installation Requirements

The installation requirements are as follows:

1. A licensed Portus Connect server must be installed with a MySQL Driver

2. A MySQL Database to hold the service data

3. A MySQL ODBC connection must be set up with the connection name ‘PortusData’ to the database to be used to back up the data services.

2.10.6 EVS Data Model Installation, Components and Configuration

This section will cover the components and configuration required to support EVS Data Model Creation.

2.10.6.1 Update EVS GUIs

Ensure you have the latest EVS GUIs deployed into your Tomcat webapps folder

The latest versions can be downloaded from the Ostia Artifactory repository:

http://cloud.ostiasolutions.com:8081/artifactory/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/com/ostiasolutions/gui

Remove the timestamps from the war files and replace the existing version of these files in the Tomcat webapps folder of your EVS installation with these latest versions.

2.10.6.2 Install Portus Connect

Note: Portus Connect Server installation and updates will require a Portus Connect License.

Download Portus Connect Control Centre:
http://cloud.ostiasolutions.com/eclipse37/windows/Portus-431-Win64.zip

Extract the archive to your preferred location (separate to the EVS installation) and follow section 1.1 – 2.1 of the linked instructions to install the Portus Server and configure the required MySQL driver:

http://cloud.ostiasolutions.com/portal/Portus-Guides/guides.html

You may also follow this installation video for Portus Connect if preferred: https://youtu.be/fFX8bApXRhE

2.10.6.3 MySQL Installation

It is recommended to install the MySQL Workbench to simplify configuration and interaction with the MySQL Server, however, this is not required if you are comfortable configuring and using MySQL from the command line.

Download the MySQL Installer: https://dev.mysql.com/downloads/file/?id=474802

Select the custom installation option:

Select the following options, we require the 32bit ODBC connector for Portus-Connect:

Install the selected elements as instructed by the MySQL Installer and install any required dependencies highlighted during the install:

Use the default for configuration:

MySQL should now be installed.

2.10.6.4 Create the portusdata Schema

In MySQL, create an empty database called portusdata:

Open the MySQL workbench and select the default local instance:

In the white space of the navigator window to the left-hand side of the screen, right click and select ‘Create Schema’ from the context menu:

Name the new schema ‘portusdata’.

2.10.6.5 ODBC Connection Configuration

Run the 32bit Data Source Admin tool:

Create a 32bit MySQL connection under the System DSN tab and name it PortusData. Fill in the connection details, selecting the portusdata schema as the default database – note that the capitalisation in PortusData is important:

Save and close the connector window on success.

2.10.7 The Process to Create a Data Model

Before using the Data Model feature, please take note of the first point below regarding the SoapUI Proxy – certain proxy settings may cause unexpected behaviour.

2.10.7.1 SoapUI Proxy

EVS uses some SoapUI functionality when creating the Data Model. If you have a SoapUI Client installed on your system, ensure that the SoapUI Proxy is turned off during Data Model creation

2.10.7.2 Create the Base Sandbox

Build the sandbox project using the Portus EVS GUI providing the appropriate transport and payload information.

2.10.7.3 Create the Base Data Model

This is done as an additional step of the GUI process to create the sandbox. Once the sandbox has been created, the configuration for the Portus Connect server must be provided. This consists of:

- The host on which the Portus Connect server is running.

- The port on which the Portus Connect service is listening.

- A setting to determine if existing services and tables found will be deleted. This can avoid overwriting existing data that has been created.

- An optional userid for the Portus Connect server if required.

- An optional password for the userid if required.

Once this has been configured. The simple click of a button will result in the data services being created.

2.10.7.4 Review and Modify Initial Data

Using the Portus Monitoring and Configuration GUI, the newly created data services may be viewed and the data for each reviewed. The GUI allows the updating, addition and deletion of data for each service to prepare for initial basic testing using the sandbox.

2.10.7.5 Start the Sandbox and Update the Configuration

The sandbox created must be started. It will by default not use the data services. Using the Portus EVS Monitoring and Configuration GUI, the configuration for the sandbox may be modified to set this up. The configuration parameters ‘dataHost’ and ‘dataPort’ must be provided to provide details of where the Portus Connect server to use is running.

2.10.7.6 Verify Operation

Once the sandbox has been started and configured, the sandbox may be called and the data returned to each request reviewed to ensure it is coming from the data services.

A second verification that is useful for simulated services is for those services that can return multiple sets of data is to find the data services related to those and to add further records. Without any further action, the next time the service is called, the extra data sets will be presented showing the new data added.

2.10.7.7 Next Steps

The next steps will involve fleshing out the sort of test cases that your developers and testers need available. This could be done by giving them access to the GUI to set up data for the service or providing an Excel spreadsheet to be filled out with the data required by the developer or tester for their particular project.

An additional step would be to further improve the sandbox by including rules to gradually ensure the sandbox correctly simulates what the real system being simulated does.

Note: In order to write the configuration to retain the details for the datahost and dataport among other configuration changes, it is required to terminate the project cleanly. For example, using jetty:stop to terminate a project running with jetty.

2.11 Portus EVS Project Management

Portus EVS is a framework that creates and maintains Maven projects which implement the functionality required in the sandboxes. These projects can be stand alone or may be sub projects of a larger project. In order to create or update the projects, a GUI is provided to initially create the project and to make changes to the project after it has been created.

2.11.1 Project Structure

Portus EVS projects are like any other Maven project and if your organization already has a standard for the structure of your project, this can and should be used to manage Portus EVS projects. It is important to have a well-defined project structure that is controlled for a number of reasons:

- It is simplest if these structures are used to commit changes to your source control system such as SVN or CVS.

- It will ensure that related sub-projects are managed and maintained in a uniform way.

- It will help with versioning of the projects.

- It will help with backup and recovery of projects.

As each Maven project is simply a directory with a fixed structure on the file system, Ostia recommend that projects are collected into related sets of sub projects. For example, if you are creating a Payments sandbox and a sandbox for your PSD2 APIs, you might consider the following structure:

./projects/payments
./projects/PSD2

As each sandbox may be made up of multiple Maven sub projects, these projects may be added to the appropriate directory above. If there is a desire to build the entire sandbox in one go, a ‘master’ pom file can be added to the higher level directory (e.g. /projects/payments/pom.xml) which can reference the sub projects and ensure they are all built together. It can also be used to ensure that the sub projects are using consistent versions of software on which they are dependent by declaring these as dependencies in the parent pom rather than in the project pom.

2.11.2 The Portus EVS Project Management GUI

When you start the GUI, you will be presented with a menu as follows:

The first thing you must do is select the directory in which you wish to work. This can be done by clicking on the ‘Select project directory’ button where you will be shown a tree structure that will enable you to select the project directory in which you wish to work as can be seen in the following screenshots:

Once selected, you can elect to create a new project or select an existing project.

To select a new project, you must provide the name of the project and the nature of the transport or protocol it will use as this will determine how you will progress through the wizard.

To select an existing project, check the ‘Existing project’ radio button, then hit the ‘Select existing project’ button and you will be presented with a list of Portus EVS projects in your selected directory. You can then select the one you wish to modify.

Once a new or existing project has been selected, you simply progress through the GUI using the ‘Next’ button on the bottom of your screen. As you move through the process, for existing projects the current definitions can be seen and modified. For new projects, you must provide the information required.

The wizard has a standard flow as follows:

1. You must provide the information for the transport or protocol selected. For example, for MQ, the queue manager and queue names must be provided, for REST the original host and port must be provided and so on.

2. You must provide the payloads that will be used in your project. For example, if you will receive JSON messages, you must provide JSON models for those messages, for XML messages, you must provide XSD definitions, for COBOL messages, you must provide COBOL structures and so on.

3. You must provide the methods that will be available in the project and the payloads they will receive and return. For MQ, Sockets and JMS, there is one method and thus you must provide the request and response message formats based on the defined payloads. For REST and WSDL, there will be normally more than one method.

4. You will then be presented with the ability to build or update the project which is the final step.

These are described in more detail in the following sections:

2.11.3 Providing Transport Information

This is where the transport or protocol specific information for each type of web service is provided. The information will be different depending on the format.

2.11.3.1 REST

The following screen will be presented:

- Set Host or IP where the real service is running. (While this is required, it will not be used unless the real service must be called.)

- Set the Port where the service is listening. (As above)

2.11.3.2 JMS

The following screen will be presented; add your JMS Queue Manager details as required:

Credentials for the JMS instance can be added by selecting the ‘Advanced Options’ buttons:

2.11.3.3 MQ

The following screen will be presented, add the details for your MQ instance as required:

Add Port, Server Connection Details and credentials by selecting the ‘Options’ button:

Note that if a new queue manager is being added, you can simply set the MQ Host and the port (or let it default) and hit the ‘Browse QNames’ button. If there is an MQ Manager running on that host and port, the MQ Queue Manager Name will be filled out while the list of available queue names will be made available in the drop down for both input and output queues.

2.11.3.4 Sockets

The following screen will be presented:

- Service Host: The host machine where the real service is listening

- Proxy Port: The port for the service to which requests will be sent

- Service Port: The port on which the real service is listening

- Request length: Length of the Request

- Response Length: Length of the Response

2.11.3.5 WSDL

The following screen will be presented:

- Enter the WSDL url and move the mouse cursor out of the ‘Source’ box to view a list of available Operations for that WSDL

- Select the operations you want to include in the service

2.11.4 Payload Definition

Portus EVS understands a number of payload formats. Payloads may be used in the following ways:

1. As the input (or request) to a specific method in the virtual service.

2. As the output (or response) from a specific method in the virtual service.

3. Payloads are also used to represent a particular context within a project which retains information from method call to method call. Please refer to the documentation describing the development of a Portus EVS sandbox for further information.

The current formats supported are:

- JSON – a JSON file or JSON schema representing the message format must be provided.

- XML – an XSD representing the XML must be provided.

- COBOL – A COBOL structure must be provided for the payload format

- RAW – No meta data is required as the virtual service implementation will simply be passed the raw data which can be used for payloads which cannot be described with meta data (e.g. payloads that are a combination of COBOL and XML)

A unique name within the project must be defined for the payload. By default, this will be the file name of any Meta data provided in the dialog. For Raw data, you must provide a payload name.

Note that it is also possible to use any combination of payloads within a single project. The only restriction is that they must have a unique ID.

To add a payload, proceed as follows:

Hit the ‘Add’ button:

Select type:

For JSON, XML or COBOL payloads, Hit the ‘Upload’ button and select payload file:

Note that for a RAW payload, you must simply give it a unique payload ID.

For COBOL Payloads, there are a number of additional configuration options. For details on these options, see Portus EVS record payload

Example of completed Payload page:

JSON Example:

COBOL example:

To delete a payload, proceed as follows:

Select the Payload you want to delete by clicking on the entry and then click on the remove button

Note: if working with an existing Project, you will need to remove the reference from any methods using that payload before removing the payload.

2.11.5 Managing Methods

In general, a virtual service will expose one or more methods that can be called. This will depend on the transport or protocol defined for the virtual service:

- For MQ, JMS and Sockets, there is one method implementation. For these methods, you simply associate a payload with the request and the response.

- For REST, there will be multiple method implementations. For PUT or POST methods, you must provide a request and response payload for each. For GET, DELETE and OPTIONS you must provide a response payload only.

- For WSDL, there will also be multiple payloads, however, as the WSDL will have defined exactly what is needed for this, Portus EVS will already have created the method definitions for you.

The following sections will document the process for each type of service.

2.11.5.1 MQ, JMS and Sockets

A Dropdown will provide a list of available Request and Response payloads based on what was uploaded during the previous payload processing step. In the following example, a single request and a single response payload are required, for simplicity these have been named Request and Response:

2.11.5.2 REST

For this rest project, a number of Request/Response payloads have been added during the payload processing stage. In the screenshots below, a number of methods are added using the corresponding payloads.

Hit the ‘Add’ Button:

Select method and related payloads from the available dropdown options:

Example of completed Methods page:

2.11.5.3 WSDL

WSDL Definitions are created for you when the WSDL URI is provided. In the method processing stage, users can define data generation parameters using inbuilt data generation functions, or add static data for the available fields.

- To the left, a list of operations and fields are presented.

- To the right, the data options are shown

To modify the data, select the field to modify from the left side, then select the ‘Change’ button to the right of the screen:

In the data window, select a function category from the left, and the function to use from the right side. Double click a function to add it to the expression window at the top of the screen as shown in the screenshot below:

Users can add static only text data, or a mix of functions and static text:

2.11.6 Build or Update the Project

This is the final step in the wizard where you will be offered the ability to build the project (when it is a new project) or update the existing project. You will be presented with a screen as follows:

For a new project, the following occurs when you hit the ‘build’ button:

1. The Portus EVS Archetype will be used to create the base project.

2. Any defined payload Meta data files will be written to the project.

3. The project properties will be written.

4. The Portus EVS Maven plugin will be run to process the payloads and create the virtual service implementation model.

This project can then be imported into the IDE of your choice and run from there.

For an existing project, the following occurs:

1. Any newly defined payload Meta data files will be written to the project. This will also result in an internal payloads.properties file being updated.

2. The project properties will be updated.

3. The Portus EVS Maven plugin will be run to process the payloads and create the virtual service implementation model. Note that this will _not_ overwrite the existing implementation as this may have changed so a new model is generated. This is to enable the actual implementation of the virtual service to be updated with any changes generated in the new implementation.

It is expected that for projects that are updated, they will already have been imported into your local IDE. In this case, following the update simply refresh the project in your IDE to pick up the latest changes.

Back to Contents

2.12 Common virtual service paths

Each different type of virtual service will follow a common path as follows:

· Read the request data.

· If callRealService=Yes:

o Call the real service.

o If response received:

§ If recording=yes

· Create a key based on the request data and the payload properties.

· Write the recording to the configured directory with a filename representing the request.

§ Return response to caller.

o If no response received:

§ If callVirtualServiceifRealServiceFails=No, return the error

§ Otherwise, proceed

· If replay=yes

o Create a Create a key based on the request data and the payload properties.

o Determine if there is a recording file for that key in the configured directory

o If yes:

§ Build the response with the recorded data.

§ If recording=yes. Write the recording to the configured directory with a filename representing the request.

§ Return the response to the caller.

· Call the virtual service implementation.

· If this returns successfully:

o If recording=yes:

§ Create a key based on the request data and the payload properties.

§ Write the recording to the configured directory with a filename representing the request.

o Return response to caller.

· If the call to the virtual service fails, return the error as the response.

Note that this is likely to change and be enhanced based on customer requirements.

Back to Contents

2.13 Data generation capability

The data generation capability in Portus is enabled in the virtual service implementation by simply adding the following Java import to the Java class:

import com.ostiasolutions.api.datagen.DataGenFunctions;

It is then possible to use the data generation functions available in this class to create data in your virtual service.

These functions are being extended based on user requirements and will be general updates.

It is also possible, of course, for organizations to simply use their own data generation classes written in Java.

Back to Contents

2.14 Hierarchy of virtual service creation

Portus has a hierarchy of virtual service creation as follows:

· First the transport is identified. This could potentially be:

o WebSphere MQ.

o Web service (SOAP over HTTP).

o TCP/IP sockets.

o REST (available October 2016).

o JMS (available November 2016).

o FTP (on request).

o APPC (on request).

o Please contact Ostia with any other transport requirements.

· Second the payload is identified. This could potentially be:

o Byte: The payload on the request is simply presented as a Java byte[] array to the service and expects the same as the payload response.

o XML: The payload on the request is simply presented as XML and the response is also expected to be XML. These are parsed by Portus using their associated XSD and presented to and returned from the virtual service implementation as a Plain Old Java Object (POJO).

o SOAP: The payload on the request is SOAP and a SOAP response is expected. This is parsed by Portus using the WSDL for the SOAP Service. This is slightly different to XML as the virtual service implementation is called with a POJO describing the request in the SOAP Body and expects a POJO in return which will integrated into the SOAP Body in the response.

o Flat record structures such as those described by the COBOL language. This are processed using jrecord and a construct is passed to the virtual service implementation to enable data in the request to be accessed using its field name and for the response to be built by setting values for each desired field name.

§ Note that while COBOL is currently the only metadata supported, other metadata can be supported on request.

Each transport, protocol and payload has different characteristics which are described in the section dedicated for each transport and payload.

Back to Contents

2.15 Portus EVS record and playback

Portus EVS offers the capability to record responses and play them back in much the same way as other frameworks. This is provided more to enable some sample datasets to be recorded rather than being used for a record and playback function which is fully supported and available within the IVS component of EVS. Ostia recommend the more flexible approach of building a true virtual implementation of the service required for best results.

2.15.1 Setting up Recording

Recordings are written to a directory called ‘<RecordingsDirectory>/<RecordingDirectory>’.

‘<RecordingsDirectory>’ is a high level directory where sets of Portus recordings may be written. This can be set from the MonitoringConfiguration GUI and will default to ‘PortusRecordings’.

‘RecordingDirectory’ is the name of a directory where a particular set of recordings will be written. This can be set from the MonitoringConfiguration GUI and will default to ‘NewRecording’. Below we see two sets of recordings with different ‘RecordingDirectory’ names, both under the high level PortusRecordings <RecordingsDirectory>.

<RecordingsDirectory> could potentially be a hard file name or a relative file name. By default, it is a relative file name and will be written to the project directory when the project is being run under Eclipse. It will be written by default to the bin directory, or otherwise the specified directory when run under Tomcat e.g.: ..\PortusRecordings will create the recordings directory one level higher in the tomcat root directory.

Eclipse:

Tomcat:

Recording will only occur when recording is set to ‘Yes’ in the run time configuration for a project which can also be set from the MonitoringConfiguration GUI.

2.15.2 Recording responses

Any record response will be recorded in <filename key>.payload file in the recordings directory. This will be the record format that would be returned on the call. While this may be modified, care must be taken if there is binary data in the payload as the editor could translate characters and thus corrupt the binary fields.

3 Portus EVS installation

Portus EVS may be delivered as a Cloud or on-premise solution. When delivered in the Cloud, Portus is pre-installed on the Cloud images that are made available for use. Therefore, these instructions are only required for an on-premise installation.

This document describes the following:

· The Portus installations required.

· The supported platforms.

· The pre-requisite software that must be installed before running the Portus installation.

· Additional resources that must be available.

· The Installation.

· The results of each installation.

Back to Contents

3.1 Portus EVS installation types required

Portus EVS has two distinct and separate installation types:

1. The ‘Power User Environment’ is where the person developing the virtual services will work. It contains all of the tools, wizards and helpers to create, modify, build and deploy virtual services. One Power User Environment is required per virtual services developer.

2. The ‘Clone Environment’ is the run time for virtual services and is the environment into which virtual services are deployed to be run and used. Clone Environments are intended to be created on demand as different users and teams require access to the virtual services created.

Back to Contents

3.2 Power User installation

3.2.1 Supported platforms

The following are the platforms currently supported by Portus:

· All levels of Windows supported by Microsoft.

Please contact Ostia if support is desired on other platforms.

3.2.2 Pre-requisite software

The following software must be installed before proceeding with the Portus installation:

· Java SE Development Kit 1.8.

· Apache Maven 3.3.9.

· If you do not use Eclipse as your IDE, you will need to install your favourite Java IDE (e.g. NetBeans etc.) for managing virtual service projects once created.

3.2.3 Other resources

Ostia deliver Portus EVS using Maven dependencies and thus a Maven repository containing the various Portus dependencies must be accessible from the machine where the Power User will run. This can be achieved as follows:

· If access is available to http://cloud.ostiasolutions.com:8081/, this is the central Ostia repository where Portus updates are made available and is the optimum configuration to ensure seamless and quick availability of fixes or updates.

· If access is not possible due to firewall rules, often organizations set up a mirror repository in their Demilitarised Zone (DMZ) which can then periodically download updates from Ostia’s site. Power Users can then use the mirror in the DMZ.

· If this is not an option, please contact Ostia to discuss the optimum potential setup for your organization.

3.2.4 Settings.xml

In order to point maven to the Ostia Artifactory repository for EVS artifacts a settings.xml file is provided to EVS users and can be found in the ‘utils’ folder under the Ostia Solutions installation directory. This settings file must be placed in the .m2 directory alongside the repository folder:

The settings.xml file can be modified to accommodate any additional requirements the organisation may have as long as the Ostia repository locations remain in place.

3.2.5 Proxy Settings

Depending on the organisation, proxy settings may need to be adjusted in order to successfully reach the Ostia Artifactory repository via maven if a DMZ mirror is not put in place. The Ostia Artifactory repository runs on port 8081 and can also be reached via browser using the following url: http://cloud.ostiasolutions.com:8081.

3.2.6 Installation

You will require the Portus EVS Power User installer and a license key to install Portus on your system so proceed as follows:

· Download or copy the Power User installer to your machine.

· Download or copy the Power User license to your machine.

Launch the installer and follow the steps in the wizard:

When prompted, add the license file:

The installation wizard will install the necessary files from the installation kit (including a Tomcat server), start the tomcat server and deploy the applications to the server.

Once the applications are ready, the landing page will open in a new browser window. From here you can access the available tools and documentation.

Once completed, please follow this online guide to install the Portus Server on this machine.

3.2.7 The results of the installation

At the completion of the Power User installation, the following will have been installed in the Power User Environment:

· A Tomcat instance with a landing page containing:

o The Manage Projects GUI – this is the main project management interface

o Links to each of the wizards to create virtual services. (now depreciated)

o Links to each of the wizards for data generation.

o A link to the Portus monitoring and configuration wizard.

o A link to the documentation.

o A link to a page describing how to start the Portus Control Centre.

Note: Ostia recommend that no other non-Ostia software is installed within this Tomcat instance and it is maintained exclusively for Portus use.

· The Portus Eclipse Based Control Centre.

o The Portus Server may be installed from the Portus Eclipse Control Centre.

Please proceed to the section on Clone Environment installation.

Back to Contents

3.3 Clone Environment installation

3.3.1 Supported Platforms

The following are the platforms currently supported by Portus:

· All levels of Windows supported by Microsoft.

Please contact Ostia if support is desired on other platforms.

3.3.2 Pre-requisite software

The following software must be installed before proceeding with the Portus EVS Clone installation:

· Java run time environment version 1.8. This is available for download here.

3.3.3 Other resources

Each Clone Environment needs TCP/IP access to the Power User Environment when specific services are used.

3.3.4 Installation

You will require the Portus EVS Clone Environment Installer and a license key to install Portus on your system so proceed as follows:

· Download or copy the Portus clone installer to your machine.

· Download or copy the license to your machine.

Launch the installer and follow instructions provided by the installation wizard.

3.3.5 The results of the installation

At the completion of the Clone Environment installation, the following will have been installed in the Power User Environment:

· A Tomcat instance with the following installed:

o The Portus Tomcat monitoring application.

Note: This Tomcat is intended as the target for deployment of Portus created virtual service applications. Ostia recommend that no other software is installed within this Tomcat instance and it is maintained exclusively for use by Portus virtual services.

· The Portus Server.

Back to Contents

3.4 Next steps

Once completed, Ostia recommend you spend some time understanding the concepts of how Portus creates virtual services before attempting to start creating virtual services, perhaps making use of the tutorials:

· Create a MQ COBOL virtual service

· Create a sockets virtual service

Back to Contents

4 Portus EVS Updates

The majority of application updates for EVS will be delivered through maven and will be pulled down automatically during project builds. In some instances, however, the application file will need to be updated.

4.1 Manage Project GUI Update

To update the Manage Project GUI, download latest manage project .war file from the Ostia Artifactory Repository. The most recent update will be the last .war file in the list with the most recent timestamp as part of the file name as shown in the following image:

Once selected, download the war file and rename the new file to manageProject-1.0.war, this will ensure that the application GUI can still be accessed from the EVS main landing page via the link provided.

Stop the Tomcat server before proceeding.

Delete old existing manageProject -1.0.war in the Ostia Solutions Tomcat webapps folder, and the related manageProject-1.0 directory.

Add the new manageProject war file to the webapps folder.

Start the Tomcat server and allow some time for Tomcat to expand and load the new application.

4.2 Monitoring GUI Update

To update the Monitoring GUI, download latest monitorConfig .war file from the Ostia Artifactory Repository. The most recent update will be the last .war file in the list with the most recent timestamp as part of the file name as shown in the following image:

Once selected, download the war file and rename the new file to monitorconfig-1.0.war, this will ensure that the application GUI can still be accessed from the EVS main landing page via the link provided.

Stop the Tomcat server before proceeding.

Delete old existing monitorconfig-1.0.war in the Ostia Solutions Tomcat webapps folder, and the related monitorconfig-1.0 directory.

Add the new monitorconfig-1.0.war file to the webapps folder.

Start the Tomcat server and allow some time for Tomcat to expand and load the new application.

5 Portus EVS licensing

It is important from both a supplier and a customer perspective that it is clear what is licensed and who is using those licenses. For this reason, Portus EVS implements a licensing capability for the three flavours of implementation that are available, namely:

· Power User

· Single Clone User

· Gateway Clone

This describes the various features associated with this licensing with a view to ensuring that the licensing component is as unobtrusive as possible.

5.1 Hardware lock

Each license is tied to a specific server where the software is running. When requesting a license, the MAC address for the active network card on the server where the Portus component will be running must be provided to Ostia. This can be found by running the function to display the hardware key found in the following location in the Portus EVS Installation kit:

\%USERPROFILE%\Ostia Solutions\utils\HardwareID-Viewer\LICENSE4J-HardwareID-Viewer.exe

Note that many, if not all servers now have multiple Network cards so the one that will be active when Portus is used must be chosen. For example, on a note book, the MAC address is different depending on whether you are working through a docking station, connected via cable to the RJ45 socket on the server or using the wireless capability.

5.2 Moving a license

Ostia provide for a license that is capable of being moved between machines. In this case, the license will be reissued with the new MAC address for the new machine, however, Ostia must first deactivate the license on the older machine which must be done in conjunction with the user to ensure service is maintained during the move of the license.

Back to Contents

6 Transport and protocol support

6.1 Portus EVS HTTP transport

HTTP can theoretically be sent over sockets, MQ, JMS etc. however; its most widely used implementation is over the sockets transport where it can add much more meaning to a request or response via metadata tags it can add. HTTP has proven itself to be one of the most interoperable transports over the past number of years and is the transport upon which the Internet has thrived. This describes Portus’ implementation which strictly runs over sockets - TCP/IP currently.

6.1.1 HTTP semantic

While there are a number of semantics, the request/response semantic is by far the most widely used:

· Application under test sends a HTTP request to the service listening on a well-known host and port.

· Service receives the HTTP request.

· Service processes the request and creates a response.

· Service sends a HTTP response to the application under test.

· Application under test receives the HTTP response.

Portus EVS introduces the concept of proxy HTTP services for the purposes of service virtualization. The semantic then is as follows when the real service is being called from Portus on behalf of the user:

· Application under test sends a HTTP request to Portus listening on a well-known host and port.

· Portus receives the HTTP request.

· Portus processes the request as appropriate. (I.e. manages payload etc.)

· Portus sends the HTTP request to the service listening on a well know host and port.

· Service receives the HTTP request.

· Service processes the request and creates a response.

· Service sends a HTTP response to the application under test.

· Portus receives the HTTP response.

· Portus processes the request (e.g. it may record it).

· Portus sends the HTTP response to the application under test.

· Application under test receives HTTP response.

Where the real service is not being called, the semantic is different:

· Application under test sends a HTTP request to Portus listening on a well-known host and port.

· Portus receives the HTTP request.

· Portus processes the request as appropriate.

o It may look for a recording matching the input request.

o It may call the virtual service implementation.

o It may record the response.

o It may delay the response based on configuration variables.

· Portus sends the HTTP response to the application under test.

· Application under test receives HTTP response.

As can be seen, there are potentially other uses that this can be put to, for example:

· In flight data masking of requests as they return from the real service.

· It is possible to cache responses that are only used should the real service not be available.

· It would be possible to cache responses so that the real service is never called until the recordings ‘time out’ and thus the real service may be called again. This could potentially avoid load on a service where return values only change periodically.

6.1.2 Recordings for HTTP services

When a HTTP payload is recorded by the Portus framework, there is significant HTTP metadata from the various HTTP headers that must also be recorded. This is written as an XML document to the recordings directory with the payload and will have the same filename as the payload. If the payload is to be replayed, the HTTP Headers are also reconstructed to give a 100% accurate response to the application under test.

6.1.3 HTTP service properties

When a virtual service uses HTTP as a transport, there are currently no properties defined for the current implementation as all information is derived to support SOAP from the WSDL.

Parameter	Required	Description
n/a

6.1.4 Virtual service implementation call

When the virtual service implementation is called from the Portus framework, the metadata for the request and response messages, in the form of a HttpServletRequest and HttpServletResponse respectively, is passed to the virtual service implementation along with the payload. This gives the user the opportunity to:

· Check the value of any request header passed on the request.

· Check and/or set the value of any response header to be returned on the response to the user.

This provides the utmost flexibility to the virtual service implementation.

Back to Contents

6.2 Portus EVS WebSphere MQ transport

WebSphere MQ is a messaging based architecture supplied by IBM. Originally used to access IBM mainframe systems, it is also now used extensively on Windows and Open Systems platforms today.

6.2.1 MQ service semantic

MQ has a specific messaging based semantic with the majority of services having the following semantic:

· Application under test places a request on the service request queue “serviceReqQ”.

· Service takes the request off the service request queue “serviceReqQ”.

· Service processes the request.

· Service puts the response onto the service response queue “serviceRspQ”,

· Application under test removes the response from the service response queue “serviceRspQ”.

Portus EVS introduces the concept of proxy queues for the purposes of service virtualization. The semantic then is as follows when the real service is being called from Portus on behalf of the user:

· Application under test places a request on the proxy request queue “proxyReqQ”.

· Portus takes the request off the proxy request queue “proxyReqQ”.

· Portus processes the request as appropriate. (I.e. manages payload etc.)

· Portus places the request on the service request queue “serviceReqQ”.

· Service takes the request off the service request queue “serviceReqQ”.

· Service processes the request.

· Service puts the response onto the service response queue “serviceRspQ”,

· Portus takes the response off the service response queue “serviceRspQ”.

· Portus processes the request (e.g. it may record it).

· Portus places the response on the proxy response queue “proxyRspQ”.

· Application under test removes the response from the proxy response queue “proxyRspQ”,

Where the real service is not being called, the semantic is different:

· Application under test places a request on the proxy request queue “proxyReqQ”.

· Portus takes the request off the proxy request queue “proxyReqQ”.

· Portus processes the request as appropriate.

o It may look for a recording matching the input request.

o It may call the virtual service implementation.

o It may record the response.

o It may delay the response based on configuration variables.

· Portus places the response on the proxy response queue “proxyRspQ”.

As can be seen, there are potentially other uses that this can be put to, for example:

· In flight data masking of requests as they return from the real service.

· It is possible to cache responses that are only used should the real service not be available.

6.2.2 Recordings for MQ services

When an MQ Service payload is recorded, there is significant MQ metadata from the MQMD that must also be recorded. This is written as an XML document to the recordings directory with the payload and will have the same filename as the payload. If the payload is to be replayed, the MQ metadata is also reconstructed to give a 100% accurate response to the application under test.

6.2.3 MQ service properties

When a virtual service uses MQ as a transport, the following documents the MQ related properties which will be read from the service configuration properties file.

Parameter	Required	Description
mqHost	No	Identifies the hostname or IP address where the MQ manager is running for proxy queues. Only required if the proxy queues are on a remote queue manager from the machine where the virtual service is running. Default: “”
mqPort	No	Identifies the port number on which the remote MQ manager is listening. Only required if mqHost is provided. Default: 1,414
mqQManager	Yes	This is the name of the MQ Queue manager where the proxy queues are defined. Default: none
mqServerConn	No	Identifies the Server Connection Channel on which the remote MQ manager is listening. Only required if mqHost is provided. Default: SYSTEM.ADMIN.SVRCONN
mqUserid	No	The userid for the MQ Queue manager where the proxy queues are defined when a userid is required to access the queue manager. Default: “”
mqPassword	No	The password associated with the mqUserid for the MQ Queue manager where the proxy queues are defined when a password is required to access the queue manager. Default: “”
mqInputQueue	Yes	The name of the MQ Proxy Input queue for the virtual service. Default: none
mqOutputQueue	Yes	The name of the MQ Proxy Output queue for the virtual service. Default: none
mqServiceHost	No	Identifies the hostname or IP address where the MQ manager is running for the service queues. Only required if the service queues are on a remote queue manager from the machine where the virtual service is running. Default: “”
mqServicePort	No	Identifies the port number on which the remote MQ manager for the service is listening. Only required if mqServiceHost is provided. Default: 1,414
mqServiceQManager	Yes	This is the name of the MQ Queue manager where the service queues are defined. Default: none
mqServiceServerConn	No	Identifies the Server Connection Channel on which the remote MQ manager is listening. Only required if mqServiceHost is provided. Default: SYSTEM.ADMIN.SVRCONN
mqServiceUserid	No	The userid for the MQ Queue manager where the service queues are defined when a userid is required to access the queue manager. Default: “”
mqServicePassword	No	The password associated with the mqUserid for the MQ Queue manager where the services queues are defined when a password is required to access the queue manager. Default: “”
mqServiceInputQueue	Yes	The name of the MQ Service Input queue on which the real service is waiting for requests. Default: none
mqServiceOutputQueue	Yes	The name of the MQ Service output queue on which the real service will place responses. Default: none

Note that while some mqService* definitions are required, they will not be used if the real service will not be called. This means that dummy values can be provided once there is no intention to invoke the real service. These settings can be changed later if the real service must be called in the future.

6.2.4 Virtual service implementation call

When the virtual service implementation is called from the Portus EVS framework, the metadata for the request and response message is passed to the virtual service implementation along with the payload.

In previous versions of EVS, this was simply the MQMessage received for the input request and an MQMessage structure to be used for the output message.

In the latest version of EVS, this is now a Portus EVS Class called PortusExtMQMessage. This class was introduced in support of RFH2 headers and has the following data:

- For the request input PortusExtMQMessage req:

o The MQMessage structure. This can be accessed using the getter for that field req.getMqMessage(). This can be used to check any values received in the MQMD for example.

o When a message with an RFH2 header is received, Portus EVS processes the RFH2 into an MQRFH2 class and makes this available in the provided PortusExtMQMessage. This can be access using the getter method for that field req.getRfh2().

- For the output response PortusExtMQMessage resp:

o The MQMessage structure that will be used to send the response. This can be accessed using the getter for that field resp.getMqMessage().This can be used to set desired values received in the MQMD to be used for the response for example.

o The implementation can optionally build an MQRFH2 header structure and return this to the framework using the setter resp.setRfh2(). The framework will then mark in the MQMessage structure that an RFH2 header is included and return this to the

This provides the utmost flexibility to the virtual service implementation.

Back to Contents

6.3 Portus EVS sockets transport

Sockets was used in early days to create a client/service semantic using raw sockets to connect to a service, send some data as a request and receive a response from the service at the other end. While it is unlikely that new services are being created with this transport, there are still many services out there using the transport from day to day and thus need support today.

6.3.1 Sockets service semantic

The majority of sockets services will have the following semantic:

· Application under test connects to a well know host and port where the service is listening.

· Application under test sends a request of a specific length to the service.

· Service receives the request.

· Service processes the request.

· Service sends a response using the same connection to the application under test.

· Application under test receives the response from the service.

Note in some cases, the socket is then closed and the system under test must connect again, in other cases the socket may be left open for future communication.

Portus EVS introduces the concept of proxy sockets for the purposes of service virtualization. The semantic then is as follows when the real service is being called from Portus on behalf of the user:

· Application under test connects to a well-known host and port where Portus is listening.

· Application under test sends a request of a specific length to Portus.

· Portus receives the request.

· Portus processes the request.

· Portus connects to the service host and port.

· Portus sends the request to the service.

· Service receives the request.

· Service processes the request.

· Service sends a response using the same connection to Portus.

· Portus receives the response.

· Portus processes the request (e.g. it may record it).

· Portus sends the response to the application under test over the socket on which the request was received.

· Application under test receives the response from Portus.

Where the real service is not being called, the semantic is different:

· Application under test connects to a well know host and port where Portus is listening.

· Application under test sends a request of a specific length to Portus.

· Portus receives the request.

· Portus processes the request as appropriate.

o It may look for a recording matching the input request.

o It may call the virtual service implementation.

o It may record the response.

o It may delay the response based on configuration variables.

· Portus sends the response to the application under test over the socket on which the request was received.

· Application under test receives the response from Portus.

As can be seen, there are potentially other uses that this can be put to, for example:

· In flight data masking of requests as they return from the real service.

· It is possible to cache responses that are only used should the real service not be available.

6.3.2 Recordings for sockets services

Unlike other transports, sockets is a relatively simple transport and no metadata exists about the request so the only data recorded for a sockets service is the payload.

6.3.3 Sockets service properties

When a virtual service uses sockets as a transport, the following documents the sockets related properties which will be read from the service configuration properties file.

Parameter	Required	Description
proxyPort	Yes	This is the Port on which Portus will wait for requests for this virtual service. The application under test will use the host were Portus is running and this port to connect to the virtual service. Default: none
serviceHost	Yes	This is the hostname or IP address of the machine where the real service is running. Default: none
servicePort	Yes	This is the port number on the machine where the real service is listening. Default: none
requestLength	Yes	This is the expected length of a request from the application under test. Default: none
responseLength	Yes	This is the expected length of the response to be received from the real service when a request is sent to it. Default: none

Note that while the serviceHost, servicePort and responseLength definitions are required, they will not be used if the real service will not be called. This means that dummy values can be provided once there is no intention to invoke the real service. This can be changed at a later date if the real service must be called.

6.3.4 Sockets helper functions

As discussed earlier, the sockets protocol is a very basic way to communicate and does not have the concept of a “message”. Therefore, in order to know how much data to read from a socket, Portus must know the length of the message to receive whenever an application under test connects to the proxy socket. If the request size is fixed, this is quite simple and can be specified using the requestLength property for the service.

By the same token, when the real service is called, Portus must know how much data to read from the socket for the response. Again, if the length is fixed, this can be set using the responseLength property.

In some cases, the lengths may be variable in which case the user must modify a virtual sockets helper class delivered with the project. The principle here is that when lengths are variable, generally the protocol is to include the length (or message type by which a length can be determined) in the first few bytes of the message.

Therefore, when receiving a request on the proxy queue, Portus does the following:

· Issues a “receive” for a maximum of the length provided in the requestLength property.

· The initial data read and the proposed length is passed to the readRequestLength() method in the helper class.

· This method can investigate the payload and return the correct length to receive.

· Portus will continue to receive until that length has been received and will then proceed to process the request.

When receiving a response on the proxy queue, Portus does the following:

· Issues a “receive” for a maximum of the length provided in the responseLength property.

· The initial data read and the proposed length is passed to the readResponseLength() method in the helper class.

· This method can investigate the payload and return the correct length to receive.

· Portus will continue to receive until that length has been received and will then proceed to process the request.

Note that in both cases if the length returned is too short, insufficient data will be passed to the virtual service implementation. On the other hand, if the length returned is too long, the virtual service will hang waiting for data that may never arrive and, if it does, it will mix data for different requests.

Back to Contents

6.4 Portus EVS REST transport

While technically not a standard, transport or a protocol, Representational State Transfer (REST) is a technical architecture style in common use today in application systems. Portus EVS supports the virtualization of REST services using the HTTP protocol running over TCP/IP.

6.4.1 REST verbs

The REST architecture over HTTP uses standard HTTP methods to implement a form of Create, Read, Update and Delete (CRUD) type interface as follows:

· GET: Read a resource or set of resources.

· PUT: Replace a resource or an entire set of resources.

· POST: Create a resource or an entire set of resources.

· DELETE: Delete a resource or an entire set of resources.

Note that this is purely a theoretical definition and often services do not comply with the theory and, for example, update resources on a GET request. It’s simply how some applications have evolved.

In addition, the following HTTP methods are supported by Portus EVS:

· HEAD: to simply return the headers for a GET request without any response content for the request.

· OPTIONS: represents a request for information about the communication options available on the request/response chain identified by the Request-URI.

Each method can optionally accept a payload as part of the request while some return a payload as part of the response. Portus SV supports multiple and/or mixed payloads on the requests. The payloads supported currently are as follows:

· RAW: The data is provided as supplied by the caller.

· XML: XML Documents are provided as content to the request and/or returned as content on the response.

· JSON: JSON Documents are provided as content to the request and/or returned as content on the response.

The following table summarises whether content is supported in the request or returned in the response for each supported method:

Method	Request Content	Response Content
GET	NO	YES
POST	YES	YES
PUT	YES	YES
DELETE	NO	YES
OPTIONS	NO	YES
HEAD	NO	NO

The Universal Resource Identifier (URI) provided on any REST request is a key part of any call, normally identifying the resource or resources to which the request relates.

6.4.2 REST semantic

The REST semantic is based on a request/response pair for each method:

· Application under test sends a REST request with a URI to the service listening on a well-known host and port. It optionally provides content where appropriate.

· Service receives the REST request.

· Service processes the request and creates a response.

· Service sends a REST response to the application under test.

· Application under test receives the REST response.

Portus test introduces the concept of proxy REST services for the purposes of service virtualization. The semantic then is as follows when the real service is being called from Portus on behalf of the user:

· Application under test sends a REST request to Portus listening on a well-known host and port. It optionally provides content where appropriate.

· Portus receives the REST request.

· Portus processes the request as appropriate. (i.e. manages payload etc.)

· Portus sends the REST request to the service listening on the actual service host and port (with content where appropriate and provided).

· Service receives the REST request.

· Service processes the request and creates a response.

· Service sends a REST response to the application under test.

· Portus receives the REST response.

· Portus processes the request (e.g. it may record it).

· Portus sends the REST response to the application under test.

· Application under test receives REST response.

Where the real service is not being called, the semantic is different:

· Application under test sends a REST request to Portus listening on a well-known host and port. It optionally provides content where appropriate.

· Portus receives the REST request.

· Portus processes the request as appropriate.

o It may look for a recording matching the input request.

o It may call the virtual service implementation.

o It may record the response.

o It may delay the response based on configuration variables.

· Portus sends the REST response to the application under test.

· Application under test receives REST response.

As can be seen, there are potentially other uses that this can be put to, for example:

· In flight data masking of requests as they return from the real service.

· It is possible to cache responses that are only used should the real service not be available.

6.4.3 Recordings for REST services

When a HTTP REST payload is recorded, there is significant HTTP metadata from the various HTTP headers that must also be recorded. This is written as an XML document to the recordings directory with the payload and will have the same filename as the payload. If the payload is to be replayed, the HTTP Headers are also reconstructed to give a 100% accurate response to the application under test.

6.4.4 Recording keys for REST services

Note that as a REST virtual service has multiple methods, the recording keys specified in the service configuration are different. The following table describes how the keys may be specified for each type of request:

Configuration keyword	Description
recordingKeysGet	Contains the recording keys to be used for the GET request
recordingKeysPost	Contains the recording keys to be used for the POST request
recordingKeysPut	Contains the recording keys to be used for the PUT request
recordingKeysDelete	Contains the recording keys to be used for the DELETE request
recordingKeysHead	Contains the recording keys to be used for the HEAD request
recordingKeysOPTIONS	Contains the recording keys to be used for the OPTIONS request

Where there is content for the request type, the standard ways of building a recordings key from the content is described in the documentation for each content type. For REST services, it also is possible to specify the following:

· ~RequestPath~: This will result in the path of the URI provided to the request being used as part of the recording key for the payload.

· ~QueryVariable~: This can be used one or more times to cause the value of the query variable specified on the statement to be used as part of the recording key for the payload.

6.4.5 REST service properties

When a virtual service uses REST, the following documents the REST related properties which will be read from the service configuration properties file.

Parameter	Required	Description
serviceHost	YES	This is the host name or IP address where the actual REST service is running
servicePort	YES	This is the port number on which the actual REST service is waiting for requests.

6.4.6 Virtual Service implementation call

When the virtual service implementation is called from the Portus framework, Portus calls a unique function within the virtual service implementation for each of the types of call (e.g. GET, POST, PUT, DELETE, HEAD and OPTIONS). As part of this call, the metadata for the request and response messages, in the form of a HttpServletRequest and HttpServletResponse respectively, is passed to the virtual service implementation along with the request payload (if any). The URI for the request is also available from the HttpServletRequest provided. This gives the user the opportunity to:

· Check the value of the URI, any request header passed on the request or the request payload if any.

· Check and/or set the value of any response header to be returned on the response to the user.

This provides the utmost flexibility to the virtual service implementation.

Back to Contents

6.5 Portus EVS JMS transport

JMS is a part of the Java Platform, Enterprise Edition, and is defined by a specification developed under the Java Community Process as JSR 914. It is a messaging standard that allows application components based on the Java Enterprise Edition (Java EE) to create, send, receive, and read messages. It is used extensively today for applications exchanging messages across heterogeneous systems.

6.5.1 Different JMS implementations

There are many different implementations of the JMS standard including open source and commercial products. In theory, these should work seamlessly if they have implemented the standard correctly. Ostia have implemented using the AMQP protocol but support for other protocols is available on request.

6.5.2 JMS capabilities

JMS offers two quite different ‘Messaging Domains’:

1. The Point-to-Point Messaging Domain:

· A point-to-point (PTP) product or application is built on the concept of message queues, senders and receivers. Each message is addressed to a specific queue and receiving clients extract messages from the queues established to hold their messages. Queues retain all messages sent to them until the messages are consumed or expire.

2. The Publish/Subscribe Messaging Domain

· In a publish/subscribe (pub/sub) product or application, clients address messages to a topic, which functions somewhat like a bulletin board. Publishers and subscribers are generally anonymous and can dynamically publish or subscribe to the content hierarchy. The system takes care of distributing the messages arriving from a topic’s multiple publishers to its multiple subscribers. Topics retain messages as long as it takes to distribute them to current subscribers.

The PTP domain is most appropriate for Portus EVS virtualization as it represents the more common type of applications an organization would wish to virtualize.

Pub/sub has limited support which can be extended based on a use case where virtualization can be helpful.

6.5.3 JMS PTP service semantic

Note that JMS documents and implementations use the terms Queue and Destination interchangeably. For the sake of brevity, this document refers to queues only, however, the same can apply to a JMS destination.

PTP products or applications have a specific messaging based semantic with the majority of services having the following semantic: