Saturday, March 15, 2008

Propagating Identity Information with Thread Local Variables


Often in our web applications we need to pass the logged in user identity down to business and data layers. This is mostly required when the business processes need to behave differently for different users/roles.

Straight forward mechanism would be to pass user identity as a parameter for every business process method, but can quickly become verbose and painful.

Another option is to use 'session based singleton' to store the user identity. Once the user is authenticated we use this session based singleton to store the user identity. Business processes can access this session based singleton to obtain the same. Even though this works, the bi-directional dependency introduced (as explained below) limits the usefulness of this approach.

Session based singleton uses the http-session for storing/retrieving information. i.e it has a dependency upon the web ui libraries. The business processes depends on the session based singleton to obtain the user identity. This makes the business processes to be dependent on web ui libraries. As naturally web uis are dependent on the business layer this causes a bi-directional dependency between web ui and the business layer.

Use of thread local variables presents a viable alternative design to achieve the requirement. The thread locals stores information local to a particular thread. Variables placed by one thread is can be accessed only by the same thread (Each thread can have a separate value for the variable).

Remaining section of the post looks in to the details of a thread local variable implementation for achieving the user identity propagation in Java. Mainly there are two parts of the implementation, a servlet filter and custom 'ThreadContext' class.

Servlet filter is responsible of protecting the web application from unauthenticated access. Any request without a user identity (IUser instance) in the session are directed to "login.jsp" which places a user identity in the session after successful authentication. What is important for our discussion is the fact that this filter is also responsible of attaching the user identity to the request thread (see below).
public class AuthFilter implements Filter {

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) {
HttpServletRequest httpServletRequest = (HttpServletRequest) request;
HttpServletResponse httpServletResponse = (HttpServletResponse) response;

// get the user from session
IUser user=(IUser)httpServletRequest.getSession().getAttribute(IUser.class.getName());
if (user == null ) {
//user is not in the session, not authenticated
httpServletResponse.sendRedirect("login.jsp");
} else {
//Authenticated, lets store the user in the thread context
ThreadContext.setUser(user);
try {
// call rest of the filters
chain.doFilter(request, response);
}finally {
// remove the thread local variable
ThreadContext.setUser(null);
}
}
}

public void init(FilterConfig filterConfig) throws ServletException {}
public void destroy() {}
}


In the above code if the user is authenticated then we call "ThreadContext.setUser(user)" to save the user information to the current thread local store. Once the processing is over (i.e after chain.doFilter()) we remove the user from the current thread bu calling "ThreadContext.setUser(null)".

Now lets look in to the other part of the implementation, i.e ThreadContext class. Not much of coding is required here.
public class ThreadContext {
private static ThreadLocal threadLocalUser = new ThreadLocal();

public static IUser getUser() {
return threadLocalUser.get();
}
public static void setUser(IUser user) {
if (user == null ) {
threadLocalUser.remove();
}
threadLocalUser.set(user);
}
}

You can see the static variable "threadLocalUser" is responsible of storing and managing the user identity for all the threads. Depending on the thread which calls the methods of the above class, appropriate IUser instances are selected by the "ThreadLocal" class.

Even though the thread local variables are really powerful construct you need to be very careful in using them. If the variables are not cleared at the end of the process (finally block) that memory can be leaked. Also this should not be used as an alternative to parameter passing but should only be used for contextual information propagation.

One other cool use of this would be to propagate transactions in declarative transaction management frameworks. I'm planning to release an new version of "Easy Data Access Framework" which makes use of thread local variables that completely hide the transaction instances from the developers.

Microsoft .NET framework has built-in support for passing the user identity through the thread. "System.Threading.Thread.CurrentPrincipal" can be used to get or set the user identity to the current thread.

Monday, January 07, 2008

Eurocenter BI Framework White Paper

We at Eurocenter were working hard to build few middleware frameworks throughout the past year. One of the most popular is 'Eurocenter Business Intelligence - ECBI' framework. We already have few customers using ECBI framework and are expecting to get in to the market strongly this year.

I was writing a product white paper on our ECBI framework during last week (Thanks Uchitha and Samudra for reviews). I thought of sharing the first draft through my blog. You can access the full draft here. Following is the introductory chapter of the white paper.

Challenge for Today’s Business Intelligence

Historically business intelligence was considered as a largely manual, back office work performed by high tech professionals. Data from different sources was periodically integrated in to a set of defined reports and forwarded to business users for their decision making.

Due to the ever increasing competition, the business world has demanded the technology to deliver much dynamic information for faster/better decision making. Over last 5 years business intelligence feature set had rapidly grown and the definition of business intelligence shifted to mean much more front office related activities by the information end users. During the period, numerous products appeared in the industry delivering lot of advanced and sophisticated features.

Although these products empowered users to make better decisions, most of the business intelligence products have unfortunately ignored an important fact. That is, ‘80 percent of the information users in a typical organization are often non-technical decision makers’. The complications introduced by the advanced feature set make these products to be overly difficult for end decision makers. According to our observations, it is only less than 20 percent of the available features are effectively used by business users in day-today decision making.

Evidently, the well-known 80-20 rule has a negative struck on the business intelligence product domain too. Overcoming this problem is becoming one of the most important challenges for BI products today. We believe that the business intelligence industry is now at its next maturity transition which can take the products to focus on ‘effective features’ by getting out of ‘features chaos’.

Proper channeling of right features to right users for right decision making is what is expected from the next generation business intelligence products. As a result, organizations are on the lookout for simpler and more focused BI products from the marketplace today.


Note: Credit for building this fantastic framework should go to Eurocenter BI Team lead by Samudra (Samudra, Ravith, Hiran, Eranga, Prashanthan, Vindya, Prasad, Janith, Lalinda, Rajive, Sandrina).

Sunday, December 30, 2007

Conserns for large scale integration architectures


During last couple of days I was writing a report on some existing academic research papers as a part of my postgraduate studies. I selected to write on one of my favorite topics, 'Integration Architectures'. Following is an abstract (Introduction Chapter) of my report. You can access the full report here.

ABSTRACT:

Most of the organizations today live by the slogan “Buy the best, build the rest”. Cost and risk of building from ground-up has made most organizations to consider buying commercial-off-the-shelf (COTS) products. Effective integration of these purchased IT systems has become the main responsibility of IT managers today.

Unfortunately, it is not very common for an organization to find all the required systems to be homogeneous. Often they are from different vendors having diverse architectures and operating on different platforms. The schema of the information model can widely differ from one system to another. At all these complexities, a seamless integration and smooth business process flow is what expected from the IT infrastructure.

According to Pollock (2001), robust integration architecture should support both ‘Application Integration’ as well as ‘Information Integration’ against heterogeneity. ‘Application Integration’ is the process of linking different software systems to become a part of a larger system. This is the technical solution that decides the level of integration (data level, application level, transaction level, process level, or human level) and technology of communication. Therefore ‘Application Integration’ mainly deals with the transportation of data/objects/messages between heterogeneous systems.

On the other hand, ‘Information Integration’ deals with the meaning and semantics of the communication. The meta-data, business rules and domain schema of one party should be understood by the other party for the integration to be successful. Maximum exchange of meanings by transformation of one entire domain representation schema to another partially compatible domain representation schema is the challenge of ‘Information Integration’.

‘Application Integration’ aspects are primary requirements that need to be satisfied by any integration architecture. But that is still only half of the total picture. Most integration architectures fall in to the trap of focusing on much of these technical aspects but forget the quality aspects of ‘Information Integration’. Simple integration requirements may be full filled by architectures biased to one arena, but complex integrations definitely require lot of attention and balance of both these aspects.

Despite the number of integration technologies and patterns exists, the extent to which the above goal is realized is debatable. This report looks in to and evaluates two such integration architectures published in order to solve the integration puzzle. The selected two architectures present two dissimilar approaches towards enterprise integration. Main focus of the evaluation is to study the two architectural patterns to assess their support for ‘Application Integration’ and ‘Information Integration’ aspects as set by Pollock (2001) in his white paper.


Tuesday, July 31, 2007

Structuring .NET Windows Installations

How to structure the application installation directories and how the assemblies should be placed becomes much important when you have a suit of applications with several related apps. For this type of situations, it is very natural to have few core assemblies common for more than one app.

If you come from COM background, one immediate solution that come to mind may be to store in the GAC (Global Assembly Cache). But if you are coming from Java background (like me) then you may probably like to have your application to have minimum windows dependencies. The you would prefer a application which works simply when you copy the files to a destination folder. (still GAC has it's own share of advantages like assembly versioning)

If you want a non GAC solution, design your solution installation folder to support holding the common executables. You may have your installation structure as following:

Here all the executables are placed in the root installation directory and dependencies are placed in sub directories. Then in .config files you should specify the dependency probe paths:
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="Common_DLLs"/>
</assemblyBinding>
</runtime>
</configuration>

But what if you want a much more modular structure with every application having a separate directory but common dlls are located in a separate dir, then you may have the following structure

In this case, "<probing privatePath="Common_DLLs"/>" will not work as now the dependencies are not stored in a sub directory of your executables. You have to use "<codeBase/>" directive in your configuration files. One important thing to note is that you have to make your dependency assemblies strong named in this case. Have the following in your .config files.

<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="Common_DLL_1" culture="neutral" publicKeyToken="96c6ab020ca54674"/>
<codeBase version="1.0.0.0" href="..Common_DLLsCommon_DLL_1.dll"/>
</dependentAssembly>
<dependentAssembly>...</dependentAssembly>
</assemblyBinding>
</runtime> </configuration>

If I had to select between, I would prefer second approach to the first one as this allows me to better structure my solution installation. Also strong naming can help avoiding much of the bad 'dll hell' experiences.

There are more interesting stuff to discuss here such as how you manage common configurations (in .config files) across these related applications. For an example if all the applications are connecting to a common database, then how do we keep connection string in a single location is a problem to solve (If you do not want to store in the Windows Registry either). I'm hoping to discuss this on a separate post.

Saturday, June 30, 2007

Selling or Consulting?

Today is the last day of my 7 day visit to Norway for meeting 4 of our customers. We have managed to successfully conclude every meeting resulting a new project to us :). We were mainly focusing on presenting our new reporting and business-intelligence(BI) frameworks to the prospective customers.

In one of the meetings I learnt one important lessons for my career.

We met a customer who has already purchased one of the world's leading BI tools. Even though our BI framework was better in terms of simplicity, portal features and return on investment; the product our customer has purchased had lot more features and power. Anyway we were able to convince the value addition of our tool making it difficult for him to take a decision on whether to buy our tool or to continue with what he purchased.

But for me I was feeling uncomfortable as I actually believed that it makes his systems complicated if he buys our tool, as he has already purchased a tool for his organization. I step out of my sales mode and informed him that I honestly think that he should not buy our tool but should stick to what ever he already purchased. I also presented some new cool products that are recently released by that BI product company.

Even my colleague was bit surprised to see me doing that. But that resulted a very open discussion as trust level has well improved. He wanted us to start working on a data warehousing solution for his company which will be a much bigger project compared to BI implementation. We have opened up a new business direction for our company as data warehousing seems to be promising with possible "Data Explosions" in the industry. I ended up as a happy techie since I think that was the correct thing to do to advice him not to buy out tool.

This incident opened my mind to think between "Selling" and "Consultancy".

Saturday, April 21, 2007

Success in your Organization

Being successful in their organization is every employee’s dream. There are few advices I normally provide to the juniors coming in to leadership roles in our organization. For me these qualities have helped a lot during my carrier.

  • Believe the organization: If you are hoping to be in the core team of your organization, then you must have a good trust towards the organization. If your heart does not trust the organization, then you are not in the correct place to act a leadership role. Either you give up being a leader or update your CV. When I mean trusting that doesn’t mean you should agree to every action your organization takes. But you should be certain that the organization’s acts are genuine and done with a good will.
  • Plan to grow with the organization: There is common mistake I see from most of the youngsters. They plan their growth decoupled from the organization’s growth. My advice is that you should contribute the organization to grow then normally you grow in a much better rate within the organization.
  • It is not the technical things that matters most: Many of my employees I see running behind the technologies blindly. But in my experience, the one who climb the ladder is not the best techie, but the one who is the most valuable in his context. For an example look at a team whose responsibility is to deliver a project. The one, who builds the team spirit, help others, takes responsibility, troubles less and make the delivery to happen is the most valuable one and will be given the best promotion. So never fall back if someone is more technical than you are! Technology is not the key to success.
  • Build and trust your second level: If you do not build your second level to take over from you, you will never be promoted, because no one is there to do your job. You have to start delegating your work as much as possible. You may not get the quality you expect at first, but close one eye and continue delegating. After sometime they will start performing better that you. Let your management style to be delegate-train-review.
  • Manage frustrations: As a leader never discuss or communicate your frustrations to your team. Bring up frustrations you have to your superior but not to the team. Also solve your team’s frustrations as soon as possible if any. I have seen leaders failing just because they spread their frustration to the team and then end up with a highly de-motivated team which doesn’t perform.
  • Never challenge other’s ego: There are times you need to put your foot down and be hard on your team. In such situations always go by logics and facts, never challenge the others ego. For an example if you say “How many times I have told you this? Why can’t you grasp this simple concept?” that may hurt the listeners ego. Instead you may explain the bad consequences of him not listening to you well and help him to improve the situation.
  • Argue but don’t fight: From my view point, the difference between a fight and an argument is how you conclude the discussion. If both of you end the conversation with respecting each other that will be an argument. But if any of you are hurt at the end of the conversation, then that is a fight. The good news is even if you fight with someone still you can end it as an argument before you finish with it.
  • Identify your greatest asset: Some of the leaders are afraid of their subordinates who are smarter than themselves. But they are the greatest assets you can use to climb your ladder. You may effortlessly get them to play your role and start acting at one level above your current role. This is the easiest way to grow. Never be afraid to hand over your current role to a smarter one in your team, because that is the best thing that can happen to an effective leader.

Monday, February 05, 2007

Software Architecture Attribute Checklist

After using RUP templates for software architecture specification for few years, we at Eurocenter have decided to build our own set of templates for Software Architecture and for Software Design specifications. The new templates needed to provide enough meta-information to the author for better analyze the system as well as not to miss any important design aspect. There I did some research to identify as much as possible architectural attributes (checklist) which may be important for a designer to consider in developing architecture. Before dive in to the checklist, as a background information let me explain the design documentation we prepare at various stages of a project.

In fact practically within Eurocenter we use 4 types of design documentations. At very initial stage of the project (proposal and scope stages) we develop ‘Architecture Overview Document’ which presents the very abstract view of our recommended architecture to solve the customer problem as well as several alternative architectures with merits/demerits. This is where we demonstrate our understanding of the problem to our customers (this is particularly helpful in our presale activities).

The next document on the pipeline is the ‘Software Architecture Specification’, describing the meta-structure of all software structures. Our approach in this document is based on 4+1 views described by Philippe Kruchten.

Next we perform a detail design analysis based on UML notations to bridge the gap between system architecture and implementation. We call this document the ‘Software Design Specification’.

The final design document we prepare is called ‘Developer Guideline Documentation’ which serves the purpose of documenting miscellaneous guidelines for the developers. This section may include some best practices, version controlling guide lines, project specific knowledge base, etc. This practice is very close to maintaining a project Wiki, but much organized and get delivered to the customer for reference at the end of the project (note that all our customers are software organizations).

Now, enough back ground. Let’s have a look at the key attributes in developing a software architecture specification. You may use the following as a checklist to verify that you do not miss any important architectural aspect. Basically these are related with the non-functional goals of your architecture. Perfect system architecture may describe the expected level and realization strategy for each of the following attributes.

  • Performance
    • Response time
    • Throughput
    • Scalability (supporting increasing loads – load balancing)
  • Operational
    • Availability
    • Manageability (How to manage executing components like Caches)
    • Upgradeability
    • Reliability
    • Recoverability (Fault Tolerance)
    • Flexibility (Ability to support multiple configurations, workflows, etc)
    • Transparency (Hide the complexities)
    • Distribution, Concurrency and Conflict resolution
    • Integration (Connectivity to other systems)
    • Resources (Constraints and requirements)
    • System configurations
    • Offline Operations
    • Stability, Consistency and Accuracy
  • Maintainability
    • Portability
    • Complexity
    • Understandability
    • Duplication
    • Fragility (possibility of breaking the system due to a change)
    • Extensibility
    • Debugging
  • Security
    • Integrity
    • Authentication
    • Authorization
    • Safety (System may not cause the Monitor to explode)
    • Secrecy
    • Accountability (who did what and when)
    • Verifications and Validations
  • Other
    • Internationalization
    • Configurations
    • Testability (No entity beans, lets use Hibernate)
    • Usability (effective HCI)

I hope this list may help you in designing and documenting your new architectures. If I have missed any attribute let me know so that I can add those to the list. Happy architecting…