STATE STANDARD OF THE UNION OF THE SSR

INFORMATION TECHNOLOGY

TYPES OF TESTING AUTOMATED SYSTEMS

GOST 34.603-92

USSR STANDARDIZATION AND METROLOGY COMMITTEE

Moscow

STATE STANDARD OF THE UNION OF THE SSR

Introduction date 01.01.93

This standard applies to automated systems (AS) used in various types activities (research, design, management, etc.) P.), including their combinations created in organizations, associations and enterprises (hereinafter referred to as organizations)

The standard establishes the types of tests of the AU and the general requirements for x conducting.

Terms used in this International Standard and their definitions ate eniya - in accordance with GOST 34.003.

The requirements of this standard, except pp., , , are mandatory, the requirements of paragraphs. , , - recommended.

1. GENERAL PROVISIONS

1.2. NPP testing is a process of checking the performance of the specified functions of the system, determining and verifying compliance with the requirements of the TOR of the quantitative and (or) qualitative characteristics of the system, identifying and eliminating shortcomings in the actions of the system, in the developed documentation.

1.3. For the AU, the following main types of tests are established:

1) preliminary;

2) trial operation qi I;

3) acceptance.

Notes:

1. Allowed additionally holding friend x in d ov AC testing their parts.

2. Classification allowed aci acceptance test in and depending on the status of the acceptance committee (the composition of the members of the committee and the level of approval).

3. Experienced neither The status of the acceptance commission is set in the contract and (or) TK.

1.4, Depending on the interconnections of the objects tested in the AU, tests can be autonomous or complex.

1.10. Trial operation of C is carried out in order to determine the actual values ​​of the quantitative and qualitative characteristics of the NPP and the readiness of personnel to work in the conditions of operation of the NPP, determine the actual efficiency of the NPP, and correct (if necessary) documentation.

1.11. Acceptance tests of the AU are carried out to determine the compliance of the AU with the terms of reference, assess the quality of trial operation and resolve the issue of the possibility of ie MK AS in permanent operation.

1.12. AC acceptance tests should pr units to allow its experimental operation at the facility.

1.13. Depending on the type of requirements imposed on the AU and tests, verification or certification, it is subjected to:

1) a set of software and technical means;

2) personnel;

3) operational documentation regulating the activities of personnel during the operation of the NPP;

4) AS in general.

1.14. During tests, the speakers check:

1) the quality of the execution of automatic functions by a complex of software and hardware tools all modes f un To positioning AC according to the statement of work was created ie AC;

3) the completeness of the documentation contained in the operational documentation for the specified personnel for the execution nen and m functions in all modes of operation With the consent of the statement of work on the creation of the AU;

4) quantitative and (or) qualitative characteristics fulfillment automatic and automated functions of the AU in accordance with the statement of work:

5) other properties of the AU, to which it must correspond according to the TOR.

2) complex.

2.2. A offline testing

2.2.1. Autonomous tests of the AU should be carried out in corresponding vie with the program and methodology of autonomous tests developed for each part of the AU.

2.2.2. The program of autonomous tests indicate:

1) list of functions to be tested;

2) description of the relationship of the test object with other parts of the NPP;

3) conditions, procedure and methods for conducting tests and processing results;

4) acceptance criteria for parts based on test results.

An offline test schedule should be attached to the offline test program.

2.2.3. Prepared and coordinated tests (test cases) at the stage of autonomous testing should provide:

1) a complete check of the functions and procedures according to the list agreed with the customer;

2) the required accuracy of calculations, established in the TOR;

3) verification of the main temporal characteristics of the functioning of software (in cases where this is significant);

4) checking the reliability and stability of the functioning of software and hardware.

2.2.4 . As initial information for the test, it is recommended to use a fragment of real information of the customer organization in an amount sufficient to ensure the necessary reliability of the tests.

2.2.5 The results of autonomous testing of parts of the AU should be recorded in the test reports. The protocol must contain a conclusion on the possibility (impossibility) of admitting a part of the NPP to complex tests.

2.2.6. In the event that the autonomous tests carried out are found to be insufficient, or a violation of the requirements of the regulatory documents on the composition or content of the documentation is revealed, the specified part of the AU can be returned for revision and appointed new term tests.

2.3. Complex tests

2.3.1. Comprehensive testing of the AU is carried out by performing complex tests. The test results are reflected in the protocol. The work is completed with the execution of the acceptance certificate for trial operation.

2.3.2. The program of integrated testing of the NPP or parts of the NPP indicates:

1) list of test objects;

2) the composition of the submitted documentation;

3) a description of the relationships being tested between the test items;

4) the sequence of tests of NPP parts;

5) the procedure and methods of testing, including the composition of software and equipment necessary for testing, including special stands and test sites.

2.3.3. For complex testing, the following must be submitted:

1) integrated testing program;

2) conclusion on autonomous testing of the relevant parts of the AU and elimination of errors and comments identified during autonomous testing;

3) complex tests;

4) software and hardware and related operational documentation.

2.3.4. In complex tests, it is allowed to use as initial information obtained from autonomous tests of parts of the NPP.

2.3.5. The comprehensive test should:

1) be logically linked;

2) to ensure the verification of the performance of the functions of the NPP parts in all modes of operation established in the ToR for the NPP, including all connections between them;

3) provide a check of the system's response to incorrect information and emergency situations.

2.3.6. The integrated test protocol should contain a conclusion on the possibility (impossibility) of accepting the NPP for trial operation, as well as a list of necessary improvements and recommended deadlines for their implementation.

After elimination of shortcomings, repeated complex tests are carried out in the required volume.

3. PILOT OPERATION

3.1. Trial operation is carried out in accordance with the program, which indicates:

1) the conditions and procedure for the functioning of parts of the AU and the AU as a whole;

2) duration trial operation, sufficient to verify the correct functioning of the AU when performing each function of the system and the readiness of personnel to work in. operating conditions of the AS;

3) the procedure for eliminating deficiencies identified during trial operation.

3.2. During the trial operation of the AU, a work log is kept, in which information is entered on the duration of the AU operation, failures, failures, emergencies, changes in the parameters of the automation object, ongoing adjustments to documentation and software, and adjustment of hardware. The information is recorded in a journal with the date and responsible person. The journal may include comments from personnel on the ease of operation of the AU.

3.3. Based on the results of trial operation, a decision is made on the possibility (or impossibility) of presenting parts of the NPP and the system as a whole for acceptance tests.

The work ends with the execution of an act on the completion of trial operation and the admission of the system to acceptance tests.

4. ACCEPTANCE TESTS

4.1. Acceptance tests are carried out in accordance with the program, which indicates:

1) a list of objects allocated in the system for testing and a list of requirements that the objects must comply with (with reference to the points of the TOR);

2) acceptance criteria for the system and its parts;

3) conditions and terms of testing;

4) means for testing;

5) names of persons responsible for conducting tests;

6) test methodology and processing of their results;

7) a list of documentation to be drawn up.

4.2. For acceptance testing, the following documentation must be presented:

1) terms of reference for the creation of the AU;

2) act of acceptance for trial operation;

3) work logs of trial operation;

4) act of completion of trial operation and admission of the NPP to acceptance tests;

5) program and test methodology. Acceptance testing should be carried out at a functioning facility.

4.3. Acceptance testing should primarily include verification of:

1) the completeness and quality of the implementation of functions at standard, limiting, critical values ​​of the parameters of the automation object and in other operating conditions of the AU specified in the statement of work;

2) fulfillment of each requirement related to the system interface;

3) the work of personnel in an interactive mode;

4) means and methods for restoring the operability of the AU after failures;

5) completeness and quality of operational documentation.

4.4 . Verification of the completeness and quality of the performance of the functions of the AU is recommended to be carried out in two stages. At the first stage, individual functions (tasks, task complexes) are tested. At the same time, they check the fulfillment of the requirements of the TOR for functions (tasks, task complexes). At the second stage, the interaction of tasks in the system and the fulfillment of the requirements of the TOR for the system as a whole are checked.

4.5 . By agreement with the customer, the verification of tasks, depending on their specifics, can be carried out autonomously or as part of a complex. It is advisable to combine tasks when checking in complexes, taking into account the commonality of the information used and internal connections.

4.6. Checking the work of personnel in an interactive mode is carried out taking into account the completeness and quality of the performance of the functions of the system as a whole.

Subject to verification:

1) the completeness of messages, directives, requests available to the operator and their sufficiency for the operation of the system;

2) the complexity of the dialogue procedures, the ability of personnel to work without special training;

3) the reaction of the system and its parts to operator errors, service facilities.

4.7. Checking the means of restoring the health of the AU after computer failures should include:

1) checking the presence in the operational documentation of recommendations for restoring operability and the completeness of their description;

3) operability of means of automatic restoration of functions (if any).

4.8. Verification of the completeness and quality of operational documentation should be carried out by analyzing the documentation for compliance with the requirements of regulatory and technical documents and TOR.

4.9. The test results of the objects provided for by the program are recorded in the protocols containing the following sections:

1) the purpose of the tests and the number of the section of the requirements of the TOR for the NPP, for which the test is carried out;

2) the composition of the hardware and software used in the tests;

3) an indication of the methods in accordance with which the tests were carried out, processing and evaluation of the results;

4) test conditions and characteristics of the initial data;

5) storage facilities and access conditions to the final testing program;

6) generalized test results;

7) conclusions about the test results and the compliance of the created system or its parts with a certain section of the requirements of the TOR for the NPP.

4.10. Test reports of objects throughout the program are summarized in a single protocol, on the basis of which a conclusion is made about the compliance of the system with the requirements of the technical specifications for the NPP and the possibility of issuing an act of acceptance of the NPP for permanent operation.

The work is completed by the execution of the act of acceptance of the NPP into permanent operation.

INFORMATION DATA

1. DEVELOPED AND INTRODUCED by the Technical Committee TC 22 " Information technology", Subcommittee PC 052 "Automated systems"

DEVELOPERS

I.P. Vakhlakov, Ya.G. Vilenchik, F.R. Otter, cand. tech. sciences; L.M. Seidenberg, cand. tech. sciences; Yu.B. irz, cand. tech. sciences; V.G. Ivanov, V.D. Kostyukov, cand. tech. sciences; V.G. Mikhailov, cand. tech. sciences; N.V. Stepanchikov

Preliminary tests

EIS preliminary tests can be autonomous and complex.

The program of autonomous tests indicate:

list of functions to be tested;

description of the relationship of the test object with other parts of the EIS;

conditions, procedure and methods for conducting tests and processing results;

· Criteria for acceptance of parts based on test results.

An offline test schedule should be attached to the offline test program.

Prepared and coordinated tests (test cases) at the stage of autonomous testing should provide:

Full check of functions and procedures according to the list agreed with the customer;

the necessary accuracy of calculations, established in the TOR;

Checking the main temporal characteristics of the functioning of software;

Checking the reliability and stability of the functioning of software and hardware.

The results of autonomous tests of EIS parts are recorded in the test reports. The protocol must contain a conclusion on the possibility of admitting a part of the EIS to complex tests. In the event that the autonomous tests carried out are found to be insufficient, or a violation of the requirements of the regulatory documents on the composition or content of the documentation is revealed, the specified part of the EIS may be returned for revision and a new test period is assigned.

Complex tests EIS is carried out by performing complex tests. The comprehensive test should:

be logically connected;

· to provide check of performance of functions of parts of EIS in all operating modes;

· to provide check of reaction of system to the incorrect information and emergencies.

The test results are reflected in the protocol. The work is completed with the execution of the acceptance certificate for trial operation.

In the program of complex tests of EIS indicate:

a list of test objects;

the composition of the submitted documentation;

a description of the relationships to be tested between the test objects;

the sequence of testing parts of the EIS;

· the procedure and methods of testing, including the composition of software and equipment required for testing, including special stands and test sites.

The integrated test protocol should contain a conclusion on the possibility (impossibility) of EIS acceptance for trial operation, as well as a list of necessary improvements and recommended deadlines for their implementation.

Trial operation is carried out in accordance with the program, which indicates:

conditions and procedure for the functioning of parts of the EIS and the EIS as a whole;

· the duration of trial operation, sufficient to verify the correct functioning of the EIS;

The procedure for eliminating deficiencies identified during trial operation.

During the trial operation of the EIS, a work log is kept, in which information is entered on the duration of the operation of the EIS, failures, failures, emergencies, changes in the parameters of the automation object, ongoing adjustments to the documentation and software, adjustment, and technical means. Information is recorded in the journal with the date and responsible person. The log may contain comments of the staff on the ease of operation of the EIS.

Based on the results of trial operation, a decision is made on the possibility of presenting parts of the EIS and the system as a whole for acceptance tests.

The work ends with the execution of an act on the completion of trial operation and the admission of the system to acceptance tests.

The final stage in the process of project implementation software from the company 1C, is trial operation. It is necessary so that the user can check the correct operation of the system, and the developers can quickly eliminate the existing shortcomings.

What is trial operation?

In essence, this stage allows you to check the readiness of the system for its industrial use. The user, together with the developer, checks all the main algorithms of the program and, if necessary, corrects and debugs them. It is at this stage that the correctness of data processing in real conditions further operation. The thing is that some software flaws can only be identified when working with a large amount of information.

The most common flaws

In the process of using the program in real conditions, some flaws may arise due to the “unpreparedness” of the system for working with a large amount of data. Here are some of them:

• inertia, or in other words, the inaction of the system when creating some documents. To identify this error, it is recommended to check the possibility of creating each document using the maximum number of positions;
• the program takes a very long time to generate some reports. Most likely, the system is not ready to work with a large amount of data, or some algorithm does not function quite correctly. The developer needs to double-check the entire chain and fix the problem;
• appearance conflict situations when working with some objects. Generally, in most cases, we are talking about "complex" elements;
• problems with conducting documents, which are most often caused by an incorrect algorithm.

Main types of trial operation

Depending on how the software is produced, two main types of trial operation can be distinguished:

1. there are situations when the development stage is forced to “mix” with trial operation. This is relevant in those situations when it comes to a rather complex configuration that requires a large number of individual add-ons. The thing is that in the development process, it is not always possible to take into account all the requirements for the program;
2. the second type is the “standard” trial operation. It represents the simultaneous use of existing software and the implemented software product. As a rule, real databases, lists of counterparties and ongoing operations are used here. This is done so that the user can determine how correctly the program performs a particular task. Most often, in order to optimize this process, developers make it possible to temporarily exchange information between the two systems. Thus, both programs will use the same information, which will minimize the risk of an accidental error.

Each user must be prepared for the fact that at the stage of trial operation he will encounter certain shortcomings. The whole point is that base model the program has already been repeatedly tested in real conditions, while a specialized configuration is often installed in a “raw” form, verified only during the development process. As a rule, developers quickly eliminate all the shortcomings, and the program begins to function quite correctly and stably. Among other things, one should not exclude such a nuance that already existing system can be the cause of problems when testing a new product, because it is far from always possible to immediately configure the optimal compatibility and data exchange settings between systems.

Creation information system begins from the moment of the first negotiations between the Customer and the potential Contractor and may never end, as good and useful systems are constantly being improved and developed.

preliminary stage

At this stage, it is necessary to understand the main goals and objectives of the future project. To do this, key representatives of the Customer and the Contractor organize meetings at which they discuss the concept of the information system, key technical points, the timing and scope of work performed, as well as the cost and sources of financing. The result of the preliminary stage, in addition to the agreed terms of the future contract, should be the first and most fundamental project document - project charter.

The project charter defines the following fundamental points related to the process of developing and implementing an information system:

• Brief description of the project, goals and objectives of creating an information system.
• General description of the scope of work.
• Project boundaries: terms, budget, list of automation objects.
• Product description: list of supplied hardware and software, type and number of licenses, etc.
• Organizational structure of the project: the list and roles of the project team members on the part of the Contractor and the Customer, their responsibilities and duties, the project document management system.
• The main stages of development and implementation of the information system, an enlarged schedule for their implementation.
• The most significant risks of non-fulfillment of obligations under the project, as well as ways to minimize risks.

In other words, the project charter is precisely the charter that is developed by the project manager together with the main members of the project team, approved by the management of the Contractor and the Customer, and should not be adjusted during the entire time of creating the information system.

The completion of the preliminary stage can be considered the moment when the contract for services for the development and implementation of the information system is signed and the charter of the project is approved.

Gathering Requirements

At this point, all the key figures - project participants have been identified, and nothing prevents us from starting to collect and approve the requirements for the future information system. Representatives of the contractor communicate with future users and administrators of the system, as well as with their management. During the survey, not only the requirements and wishes for the implemented solution are systematized, but also the documentation is analyzed, which should become the source of the system's initial data, or the formation of which should be automated as a result.

result this stage should be the appearance terms of reference for the development and implementation of the information system. The terms of reference should be based on the terms of the contract and the requirements set forth in the project charter and contain the following sections (for Russia, the structure of the terms of reference is regulated by GOST 34.602 89):

• Purpose and purpose of creating the system.
• Description of the automation object and the main automated business processes.
• System requirements: structure requirements; functions (tasks) solved by the system; technical and organizational support; requirements for reliability, safety, etc. and so on.
• The composition and content of work on the creation of an information system.
• The order of control and acceptance of the results of work.
• Requirements for the scope of work for the preparation of an automation object for putting the information system into operation.
• Requirements for the composition of design and user documentation.

Completion of the requirements collection stage is the approval of the Terms of Reference by the Customer. In some cases, before the start of work on the project, the Customer already has a terms of reference (included in the tender documentation). In this case, the results of the survey and collection of requirements are recorded in private terms of reference, detailing and concretizing General requirements to the information system presented in the original terms of reference.

Design

At this stage, by the efforts of the Contractor, all scenarios related to the development and implementation of an information system on the territory of the Customer are designed in detail. This is done in accordance with the conditions of the information environment (system landscape) of the Customer and the requirements for the integration of the system being created with others already available and operated by the Customer. software products. The result of the design phase should be the design of the following sections technical (conceptual) project:

• Information system architecture.
• Description of information storage (database) structures.
• Design solutions presented by a detailed description of automation scenarios for all business processes affected by the implementation of the system.
• Scenarios for integrating the developed information system with external software products.
• Sources of initial data and options for the initial information content of the system.
• The concept of differentiation of access rights to data based on user roles, which determine, among other things, their powers.
• The concept of training users of the information system.

Implementation

The stage of implementation of all the requirements for the information system set out in terms of reference And technical project. During this period, the Contractor develops all the necessary software components, creates database structures, installs, configures and tests all components of the information system on its territory, simulates integration scenarios, etc. and so on. The completion of the implementation phase is confirmed by the appearance of such project documents as a manual for installing and configuring the system, program and test procedure system, as well as a database template and a register of all completed software developments.

Preparation of the information system for operation

All work of this stage is already being carried out at the Customer’s site and includes installation and configuration of all system components in the Customer’s information environment, preliminary testing, development of user documentation, user training, loading of initial data, testing in accordance with the program and methodology, and other preparatory work.

By the time all the preparatory work is completed, the operating procedure for the system should be developed and approved. The regulation, in particular, should define users and their roles in the system, in accordance with their job responsibilities.

Pilot operation

The last stage in the development and initial implementation of an information system, the task of which is to successfully conduct a trial operation of the system for a certain time, and the goal is to confirm that the created information system is exactly the result that the Customer expected.

During this period, users begin to operate the system in accordance with the regulations developed at the previous stage. During the experimental industrial operation errors are fixed and necessary improvements are agreed. The Contractor eliminates errors, performs improvements and, provided that the system begins to function in accordance with all the requirements presented to it earlier, at the end of the established period, receives a protocol on the successful completion of pilot operation.

With the completion of the pilot operation, as a rule, the contract for the creation of an information system ends. The system itself goes into commercial operation, and the Contractor, if the Customer is interested in this, concludes separate contract for its support, for the period established by the terms of the contract.

Maintenance and development of the system

Industrial operation may reveal that some requirements for the created information system contained inaccuracies and require a different wording or additions, and the system itself needs to be improved. Not every Customer has staff in its staff that is able to independently introduce all changes dictated by the real situation into the system, therefore, it concludes a separate agreement with the Contractor for the maintenance of the information system.

Users of the information system begin to communicate with representatives of the Customer's support service, who receive requests from them for improving functionality and eliminating defects, transfer requests for work, and periodically notify users of the status of their request. The list of possible improvements and the procedure for processing applications is determined by the terms of the contract. If there is a need for work that does not fit into the essence of the contract for support, then the Customer and the Contractor draw up a separate contract for this species work, which can already be attributed to the work on the modernization and development of the operated information system.

At this stage, by the efforts of the Contractor, all scenarios related to the development and implementation of an information system on the territory of the Customer are designed in detail. This is done in accordance with the conditions of the information environment (system landscape) of the Customer and the requirements for the integration of the system being created with other software products already available and operated by the Customer. The result of the design phase should be the design of the following sections technical (conceptual) project:

Information system architecture.

Description of information storage (database) structures.

Design solutions presented by a detailed description of automation scenarios for all business processes affected by the implementation of the system.

Scenarios for integrating the developed information system with external software products.

Sources of initial data and options for the initial information content of the system.

The concept of differentiation of access rights to data based on user roles, which determine, among other things, their powers.

The concept of training users of the information system.

Implementation

The stage of implementation of all the requirements for the information system set out in the Terms of Reference and the Technical Design. During this period, the Contractor develops all the necessary software components, creates database structures, installs, configures and tests all components of the information system on its territory, simulates integration scenarios, etc. and so on. The completion of the implementation phase is confirmed by the appearance of such project documents as a manual for installing and configuring the system, program and test procedure system, as well as a database template and a register of all completed software developments.

Preparation of the information system for operation

All work of this stage is already being carried out at the Customer's premises and includes installation and configuration of all system components in the Customer's information environment, preliminary testing, development of user documentation, user training, initial data loading, system test in accordance with the program and test methodology and other preparatory work.

By the time all the preparatory work is completed, the operating procedure for the system should be developed and approved. The regulation, in particular, should define users and their roles in the system, in accordance with their job responsibilities.

Pilot operation

The last stage in the development and initial implementation of an information system, the task of which is to successfully conduct a trial operation of the system for a certain time, and the goal is to confirm that the created information system is exactly the result that the Customer expected.

During this period, users begin to operate the system in accordance with the regulations developed at the previous stage. During the pilot operation, errors are fixed and the necessary improvements are agreed. The Contractor eliminates errors, performs improvements and, provided that the system begins to function in accordance with all the requirements presented to it earlier, at the end of the established period, receives a protocol on the successful completion of pilot operation.

With the completion of the pilot operation, as a rule, the contract for the creation of an information system ends. The system itself goes into commercial operation, and the Contractor, if the Customer is interested in this, concludes a separate contract for its maintenance, for the period established by the terms of the contract.