What are the architectural challenges to be kept in mind while building an EMS? And
Suggest a suitable architecture for building an Element Management System?
In a real world situation, EV's have to cater to different market segments
ranging from small Service Providers to large ILECs and the EMS requirements of each of these segments are different. A simple EMS developed for a small Service
Provider may not meet the scalability needs of an ILEC, while a full-featured highly available and scalable EMS that is suitable for an ILEC might be overkill for a
smaller Service Provider. Finally developing a new EMS for each segment may not be a practical or a profitable option.
When confronted with the above problem, the EV's
attempt to solve it by developing a basic EMS initially and hoping that it could be scaled up to meet the demanding requirements of an ILEC. This strategy definitely makes
sense if the EMS is architected properly. From our experience we' ve noticed that in many instances it is not the case because the initial development of the EMS is done at the
last moment and in a rushed manner. Therefore the EV's face an uphill task of scaling up the EMS. The key issue in most of these situations is that the EMS was not initially architected
to meet the requirements of large Service Providers. It is therefore essential to choose the correct architecture for the EMS so that it could be implemented to meet the requirements
of different segments effectively.
There are a variety of architectures that could be used to build an EMS. Based on our experience, we recommend that the EMS be developed using JEE
architecture. Since JEE based applications could be extended to n-tiers (if required) to meet the scalability needs of the customer, we believe it is a very good fit for EMS development.
What are the challenges to be kept in mind while developing an EMS to manage a device using TL1 protocol?
Managing a device via TL1 is quite different when compared to SNMP and it offers its own benefits as well as challenges.
• In simple terms, TL1 messages are text messages that have a structure (with some standard delimiters) that could be easily understood by humans. It gives the equipment vendors the flexibility to define their own command-sets provided it follows basic TL1 guidelines. This flexibility makes decoding of TL1 messages quite cumbersome since it involves parsing of a custom command set that has been defined for each device. The EMS therefore should have efficient text processing algorithms to ensure that the messages are processed quickly and efficiently. This parsing ability could limit the processing capabilities of the EMS which in turn directly affects the scalability of the EMS solution.
• TL1 protocol works over TCP which mandates a permanent socket connection to the device to receive autonomous messages from the device. (In SNMP, Traps will be sent to port 162 in the manager so it is not necessary to maintain the connection to receive Traps).
• The key issue in maintaining the TCP connection for all the devices is the management of these sessions. This involves management of all TCP socket connections and the associated threads which are required to ensure effective processing of the TL1 messages (that are received via these sockets) without dropping any of them. Since each TCP connection requires a thread and even though theoretically there is no strict limit on the number of threads that could be supported by an operating system, each thread created will lead to increase in the memory allocated for the process, limiting the scalability (as more devices will lead to more threads leading to out of memory situation quickly). Also most operating systems have some hard limits on the number of open sockets per process. Practically a single EMS server instance may not be able to handle more than 300 – 400 TL1 sessions.
• TL1 standard has support for TL1 Gateway, whereby a network element (NE) can act as a gateway for a set of NEs. An EMS can make use of the TL1 Gateway support to provide better scalability, by maintaining session with only one gateway to manage multiple devices. In general EMS solutions supporting TL1 Gateway inherently, deliver the flexibility to scale easily, when compared to others.
What is the Optimum time to start EMS development?
The time to get started on an EMS depends on several factors while we have listed the important ones, there are many more to consider depending on your particular situation.
(i) Device maturity
(ii) Customer/Market place requirements
(iii) Financial situation
(iv) Skill set availability and prioritization
As you would have noticed, of the four key factors listed above, three of them (device maturity, financial situation, skill set availability) are internal to the company while customer/market place requirement is external to the company and depends to a very great extent on the market place that you are participating. If you want to be truly a market driven company you will put item (ii) as the most important factor which we will discuss as the last factor.
As a first set step let us look at the internal factors of the company over which the company presumably has some control. Based on our experience, in general we recommend that the customer's start their EMS development when the device development is almost completed and the device is stable. This allows the MIB definition of the device to be complete and the MIBS are also stable – a key requirement for a successful and stable EMS.
The second key factor is the current financial situation, since most of the money during the device development phase are earmarked to developing the device (and rightfully so), there may not be enough funds to be allocated for the development of an EMS. This would mean that the development of a full-fledged EMS might have to be postponed for a while till additional funds are obtained. In the mean time the device has to do with a CLI or kraft interface.
The third key factor is the skill availability and prioritization. Since the EMS has a different set of drivers from your device, the skills required to do EMS development are different that from your device. If you were to do the EMS development in-house, then the EMS team needs to be staffed up and managed. There is a lead-time involved in putting together a good EMS team which should be taken into account. Managing another team is a load on the engineering management team that needs to be prioritized as well.
Lastly the most important of all the factors is the customer/market place requirement. From our experience we've noticed that some markets (especially when selling to larger service providers and carriers) and some products (high end devices and devices that need to be deployed in volume) require an EMS when the device is introduced to the market. Since a full-fledged EMS development takes about 9 to 12 months, it is advisable to understand your market requirements and start the EMS development about a year in advance before you enter the market in a big way.
What are Different Approaches in Adding Functionalities to EMS?
There are primarily two ways to get your EMS in place. The first approach is to get all the functional modules such as Fault, Configuration, Accounting, Performance, Security and Topology developed in one shot. The second approach is to get the base functionality completed and then add more functionality in phases. In the second approach the functionality most requested by the customers is put in the first phase (say Topology and Fault). In addition, the modules in phase 1 may not have all the features in them as well (i.e. the depth).
As you might have readily guessed, the first approach solves primarily for cost and time whereas the second approach solves for cash flow. Getting the EMS developed in one stretch would definitely lower the cost of EMS development and reduce the time taken to develop the EMS. Whereas the second approach is easy on the cash flow. For most startup companies since cash flow is very important (I've heard that "cash flow is the mother's milk of a start up!") it is probably better to do the EMS development incrementally.
If we were to assume that cash flow is not a problem then the next step would be to take into account other factors such as device maturity and market/customer requirements before deciding on the approach. If the device is well developed and has been stable then the first approach could be considered. Also if the market that you are addressing insists that you have a complete EMS (example ILEC and large Service Providers) then first approach makes sense. However if your device is not fully mature and you are targeting smaller customers who are not very demanding then it may make sense to go with the second option and get a basic version of EMS in place and start trials with customers (the trials will also surface many more requirements giving you the opportunity to add features that you think are appropriate to your base EMS). It is important to note that if you choose to go ahead with the incremental approach, then you need to keep an eye on your long term needs and ensure that your choice of the architecture/technology supports the incremental approach.
