A hardware problem that exists undiscovered at the time of systems programming and checkout can cost you double, or even triple. Plus, you have one angry programmer who wonders why the system and equipment has not been properly checked prior to his task.
System Test and Integration Procedures Part 4: Systems Integration Procedures
By George Wilkinson
In Part 3 of this series the Equipment Setup and Functional Testing Procedures (SETUP) was discussed. Those tests consist of simple or low-level test procedures to make certain all equipment installed is working properly in a non-integrated status (operating independently of any and all other equipment). It also provides the opportunity to do a significant amount of individual equipment equipment setup prior to the highly critical systems integration procedures described here. Plus, it can be done by a skilled and trained installation personnel before a system integrator becomes involved. This is not only seen as a cost advantage, but if problems are found it might be more convenient for installation personnel to resolve. This is a task that most do not due or even conceive, but the importance of SETUP can not be overestimated. Done properly, it can save a significant amount of time and money being forfeited to the integration task.
The Focus of Systems Integration
Every procedure up to this point has had as its focus and intended purpose on this task - Systems Integration. It has been previously pointed out that years ago systems were very simple and each device, such as audio systems or video cameras, were sold and installed by separate companies specializing in each technology. Furthermore, the systems were not integrated or connected to each other in any way. Systems today are often computer controlled, digital in nature, integrated and distributed as one technical entity. Thus, the integration of these systems take on a new dimension that did not exist previously.
Significant Characteristics of Systems Integration
Crystalizing The Task
Closely related to systems checkout procedures, systems integration is different in the way it is conducted. One reason is the result of the digital control and programming function, which is characteristic of today's systems. The systems integration process has to assume that all equipment and wiring that has been installed and is is working and wired properly before the task begins. Although troubleshooting system dysfunction or anomalies become part of the process of systems integration when discovered, these should have been discovered and resolved by all of the previous procedures. However, the reality is that despite ones best efforts to eliminate them all beforehand, some problems will occur during this task by a number of different causes.
There is also the new dimension created by software and digital control system that must be programmed for the system to operate and function as an integrated system. The programming aspect will also be discussed here, since it is usually an essential and crucial part of systems integration.
One way that might help crystalize this factor in your mind is to focus on the intent of systems integration. By definition, integration requires the functioning and communicating of various system components that are connected together in some way electronically, sometimes bridging other systems that are connected together, that all depend on one another to provide the total functionality. Therefore, the assumption that all equipment and cable installed is operating correctly shifts the focus to the primary task of systems integration. Anything else is a distraction and complicates the task of integration that can add significant time to its completion.
The systems integration task is often likely to be under duress for two compelling reasons. For one, it usually comes at the time when the project needs to be finished and the system operational by a fixed date on the schedule for the facility to open or the homeowner to move in. And second, more often than not, the timeline of the schedule has not allowed sufficient time to conduct the systems integration task appropriately.
The Timeline Dilemma
Unfortunately, everyone responsible for determining and setting the timelines for the facilities to be constructed and occupied do not allow adequate time to perform testing, programming and integration of electronic systems. Many in the construction industry have still not yet come to grips with the reality that todays electronics systems must have the final programming and testing performed after the system is completely installed. That usually can not take place until very near the end of the construction project when everything is connected and all displays and control panels are installed. The integrated aspect of these systems still allude some. The good news is that industry is slowing becoming aware of this fact and adjusting accordingly, but it is unfortunately a painful transition for many. Nevertheless, it might not ever be adequate and those are the reasons why the issues discussed here are probably forever relevant.
Equipment programming was covered in the previous SETUP procedures. Receivers, projectors, processors and equipment of all kind typically employ some type of internal programming with options supplied by the manufacture that needs to be finalized before system programming commences and not vice versa. Noting the impact of today's tightly integrated systems on the timeline, this is yet another part of the process that should be completed to the best degree possible prior to systems integration.
The Software Dilemma
Software, the instructions that are written and understood by a digital processor to control the digital processes, has been with us since the invention of the computer. Their widespread use in industry and business is not new to most of us today. Even the advent of the personal computer is now a commonly used device by nearly everyone on a daily basis. However, other devices used in commercial audio, video and audiovisual systems are relatively new and even newer in the use for home theaters and home automation systems. Touch screens are another new device in many commercial and some home installations that required both design and programming.
By its very nature, control system programming can not be effectively written unless the equipment it controls is connected to the control system to verify that the code written is correct. Once a program is written in the programming language the manufacture uses to produce the code for the controller, it is compiled and sent the controller's memory to execute. Compiled code is code that the processor understands while the programming language is something the programmer understands and writes in a simple text editor. Unless the equipment is connected to each other and the controller in the way it is to be used, it is difficult to know if the code is written correctly and the amount of testing that can be done is very limited. For this reason, the programming aspect of the project often times must be written on-site once the equipment is installed and connected. But there are ways to assist.
Pre-programming and Testing
Having to program a control system at the installation site, you are again faced with the timeline dilemma. One way to lessen this is to write as much of the code as possible before systems integration. In an ideal situation, you will be able to have access to the equipment being installed, or the same equipment on hand, to test as each portion of the code as it is being written. Hundreds of lines of code run quickly in today's processors and one piece of code can affect the way the other behaves, so testing incrementally is more efficient. The problem is this takes considerately upfront investment for the company to have equipment in early during the project and could required a significant amount of shop space plus the time to connect and configure it - not a good option in many cases. However, it is usually possible for some of the programming and checkout to be done with a little finesse in the shop. Anything that can be done to lessen the time and impact during systems integration is a good decision when possible.
Here are some practical possibilities to considered. For example, programming touch-screen controllers can often be done prior to installation. Most of them require design of the screen layouts, buttons, graphics and other elements based on the functional requirements of the project and you can test some elements such as navigation, button actions, and slide controllers on the panel itself. All you really need is the touch screen and its controller. In some instances you can set up in the shop for partial systems integration using substitute equipment for testing. For example, if you turn on a switch (contact closure) by the controller in the software, you can use a simple lamp light wired to the controller and power supply that will give you indication of that function working. In other words, it is a "simulation" of what the contact is controlling and you are just rigging it to give you visual feedback that the contact closure opens and closes in accord with the control software you wrote. In addition, you can move the same lamp to another place in the system to test other closures, one at a time, with the same setup. You might also have access to a projector or DVD player that might be available to you, which can easily be connected to test your code, assuming it is the same model or uses the same code.
Using other similar approaches it is possible to program and test a significant amount of the programming code before you have to move to the job site. The point here is, do as much of the programming as you can ahead of doing it all after the system is installed and available for you to program and test on-site. This lessens the burden on the short timeline that usually exists between when the equipment is installed, wired and tested and when the project needs to be finished and operational.
Some Important Preludes to Systems Integration
Previous Test Procedures
It is worth noting that all of procedures previously conducted from the very beginning of the project will help to minimizing the number of issues having to be resolved during the systems integration period and the resultant cost. Everything you have done previously is focused on this very moment-a moment that sometimes can be a real "pressure cooker." Each previous task lessons impact of the time crunch at the very end of a construction project and also lessens the cost of resolving those problems at this stage of the project. At a time when there can be considerable pressure on the General Contractor and you to finish and get the system working properly is not the time for dysfunctional equipment or incorrect wiring to complicate this task.
Also, you need to be keenly aware of the impact other issues have on the task of programming during systems integration. By the very nature of programming, a programmer writes code to execute routines that provide the functional requirements of a system design according to the user's requirements. In order to test the software code a programmer writes, he must test the equipment to see if it works correctly. When he executes the code and the system does not respond correctly he will assume the problem is with the code.
But what if the problem is not the code but a wiring error or malfunctioning equipment? There is usually no way to know at first, so the programmer goes about the task of going through his code, sometimes hundreds of lines of it, to find where that function is programmed and looking for the "bug" that afflicts the programming. If, however, it is a hardware problem causing the dysfunction and not the code, the programmer could spend a long time tracing down a phantom bug. Then, when he has spent all that time and not finding the problem, his next step is to start checking hardware. This becomes very time consuming.
Do you see the issue? A hardware problem that exists undiscovered at the time of systems programming and checkout can cost you double, or even triple. Plus, you have one angry programmer who wonders why the system and equipment has not been properly checked prior to his task. This is not a selfish issue, it is one of severe impact at a time in the project when you least need it.
To be fair, the problem can also be the design. Often times this is discovered by previous tests before programming and systems integration but sometimes it will not show until the later. Also, hardware is different that software. Both can make mistakes, but software is a different kind of animal in that there are all kinds of interactions that exist between groups of code and sometimes it can impart other parts without your immediately knowledge.
Note: A future article will address programming tasks and issues in more detail.
I usually do not like authors and speakers who like to tell "real-world" stories to illustrate a point because I usually find them time consuming and not interesting, but I will make on exception here.
A Case of Illustration
I was contracted by an AV company to be the project manager for a system to be installed in a commercial installation in downtown Dallas. The installation crew for the job was in place and underway racking equipment in the shop and pulling wires on-site. In this multistory office building were several fully AV equipment training rooms on various floors and a number of conference rooms. The entire system was controlled by one AMX central controller with switchers, routers and control panels located everywhere in the building. It was a massive system to just program. Although I was experienced in programming these systems myself, my plate was full and the company had wisely acquired a contract programmer the focus on that task. The installation crew and supervisor for the project had already been set by the company. As I was working with the GC and his construction schedule to build ours when I saw the stark reality: there was not nearly enough time for systems checkout and systems integration and there was nothing I could do about because the one thing I could not manufacture was time. This project got started way too late in the timeline and no one had given a thought concerning this issue by the contractor or the GC and they had no clue anything was wrong with the schedule or the inevitable consequences about to take place.
As the project neared completion, it was a Friday and the system was due to be operational by Monday. The installers were reportedly done. The programmer had been on site for three days when he called that morning in a panic to tell me he was having lots of problems, so I went to the job site to see what I could do to help. As soon as I arrived at the site the programmer, Lesley, looked directly at me and with much visible anxiety saying "George, my code is no good… in three days nothing is working!" as the installers with rags in hand were dusting the equipment and cleaning the job site, confident in their installation while the programmer had rivers of nervous sweat rolling down his face. So I tried to bring calm and rationale to the issued and responded back by telling him "Lesley, I am a programmer and systems integrator and I am here to help, so let me work with you and we will get everything working in short order." And, we had the system documentation that is needed.
The checkout started going much faster with the two of us working together. Soon our effort turned to productivity as we found each problem one by one and fixed them. Interestingly, it turned out that most of the problems were not his programming code at all, it was mostly problems related to the hardware and most of them were due to wiring errors from a substandard cowering wiring crew. After much of the system was now functioning and hunger pains had alluded us, we noticed it was nearly midnight and hungry. But we were making such good progress we both wanted to stay and get it done not matter how late we stayed. So we ordered pizza to be delivered and keep on working. It took to about 3:30 am in the morning, but we had nearly everything working as it should so we broke off for the night (morning) to come back later Saturday evening to finish up. As it turned out, 90% of the issues were hardware related and not software. When everything was fine, I looked confidently to Lesley and said "see Lesley, most all your code was good and now we have a system that works, regardless." Having to check code, then troubleshoot hardware, fix it, then check code again consumed a massive amount of time and undue stress on a programmer (and PM) that could and should have been avoided with better installers and the people and time to check out the hardware before programming and integration testing began.
This time we had it lucky. But, what if Friday night was the opening night instead of Monday? Would not be a good day. Yes, I have one of those stories also, but that is all I am going to bore you with for now. Hopefully, you see the point of how faulty hardware and the lack of a systems checkout procedure and timeline can affect systems integration exponentially.
By this time in the series you should see the importance and economic impact of everything that that needs to be done prior to systems integration. But there is one last thing...
The Significance of Documentation to Systems Integration
Last but not least to the systems integration prelude is documentation. You have heard it before, but it is always worth repeating because the significance of documentation for systems integration is as important as it gets. In fact, performing an efficient programming, systems integration and troubleshooting depends heavily on it.
The programmer should have already acquired the proper documentation early on in the project in order to know how to start to write and organized his plan for software development. The programmer needs information on the equipment that gives him what he needs to know to control the equipment, the customer's functional requirements so he knows how to design the programming and the system diagrams showing how it is all connected together. As stated in previous articles, the documentation should follow these three guide: it should be appropriate, complete, and accurate.
It is important during systems programming and integration that the documentation on hand has been "red-lined" with any changes that have taken place since originally produced and represents exactly how it was installed. "Redlines" refer to the practice of using a red pen on drawings and documents to indicate changes in the field. Redlines are eventually used to produce as-built drawings. Obviously, programmers and system integrators rely on the documentation to be up to date for final programming and systems integration. Likewise, those performing these tasks need to make sure that any changes they have made are properly documented so that accurate as-built documents can be produced at the conclusion of the project. Most contractor contracts require as-built drawings to be submitted upon completion.
Part 4: Systems Integration Procedures will continue in the next publication where the actual procedures and methodologies will be discussed.
George Wilkinson is a 35-year veteran in project management, system design, programming, testing, problem resolution, calibration and systems analysis for computer controlled audiovisual systems. He is a graduate in Electrical Engineering Technology from the University of Texas. His firm, Advanced Technological Services serves the commercial and home automation industry and he is a certified CEDIA designer. You can contact the author at firstname.lastname@example.org.
© Copyright 2006-2007 George Wilkinson. All rights reserved.
This post does not have any comments. Be the first to leave a comment below.
Post A Comment
You must be logged in before you can post a comment. Login now.