My Photo

August 2007

Sun Mon Tue Wed Thu Fri Sat
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31  

November 15, 2006

Two Useful Aspects

Sometimes it's refreshing to write a useful little aspect. Just last night I was reviewing some code that colleagues wrote for thread safety and I found this to be a really efficient way to find likely danger spots:

public aspect TrackCollectionAccess {

    declare warning: set(java.util.Collection+ *.*) || get(java.util.Collection+ *.*): "collection field access";

    declare warning: set(java.util.Map+ *.*) || get(java.util.Map+ *.*): "map field access";

}

I was able to weave this into the project and quickly review places in the code where fields containing collections and maps were being used to see if they were used in a way that wasn't thread safe. This is a nice example of a development-time aspect: using them to help you develop without using them in production.

On the other extreme, I recently extended Glassbox to allow for various kinds of application-specific plugins. There are a number of points where plugins need to interact with the underlying system, and it turns out to be a good example of how you can use aspects to modularize a feature variation. For example, here is how the DefaultPluginManager aspect allows a plugin to define new kinds of operations:

    Object around() : getOperations() {

        Collection result = (Collection)proceed();

        for (Iterator it=operationPlugins.values().iterator(); it.hasNext();) {

            OperationPlugin plugin = (OperationPlugin)it.next();

            Collection added = plugin.getOperations();

            if (added != null) {

                for (Iterator inner=added.iterator(); it.hasNext();) {

                    OperationSummary summary = (OperationSummary)inner.next();

                    summary.getOperation().setPlugin(plugin);

                }

                result.addAll(added);

            }

        }

        return result;

    }

November 14, 2006

Learning About Aspects

I enjoyed being back at a couple of No Fluff Just Stuff conferences this fall (including the Rocky Mountain Software Symposium this past weekend). Jay Zimmerman continues to do a great job of providing fresh technical content with great speakers and it's always fun to talk the audiences at these about topics like Glassbox, AOP, and AJAX.

For those who are interested in digging in to aspects more deeply, I highly recommend the Aspect Leadership Program (ALP), which will be held in Vancouver this March 14th and 15th.

The ALP is an opportunity to learn from seven of the world's leading experts, who will provide a two-day crash course on everything you need to know to figure out what aspect-oriented programming (AOP) is all about; identify whether and how you can use it to add value, drive quality and improve productivity in your enterprise; and provide practical guidance on implementing AOP. I'm also proud to be part of the presenters at this event. Please pass on the word about this to others who might be interested, too!

October 25, 2006

AOP at the Colorado Software Summit

I'm enjoying another year of presenting at the Colorado Software Summit. One of my talks is about AOP with Spring 2.0, covering both the use of Spring AOP and of AspectJ within Spring. I've really enjoyed some of the dialog with attendees when I give this talk and afterwards: it's great to hear how people are making real use of AOP and new ideas for additional uses.

At one bank, they are using Spring AOP with Spring MVC to scan incoming Web requests for SQL injection attacks. In another department they are using AspectJ to implement use cases separately such as sending email notifcations after business events, supporting state-specific regulations in a Web application, handling encrypted fields and white space trimming on Web requests. Another group is looking to detect resource failures and short-circuit requests that would use them to avoid tying up threads (while checking for restored availability periodically). This is an interesting scenario for integrating with Glassbox to detect the outages, with custom logic to manage requests when there are outages.

Another interesting use case I heard about is audting data when committing transactions using Spring AOP. It turns out you can use the Spring ordered interface to ensure the audit logic can set up just before the transaction commits (so it can be included in the transaction!). However I don't see how you specify the order of the transaction aspect in @AspectJ aspects in the relevant Spring docs as of today (Oct. 25): it is shown in the XML schema-defined .

In my talk I also give an example of how to use Spring AOP to proxy persistent objects after a DAO returns them by executing advice after they are returned, say by a finder method in a DAO. Now this isn't as nice as using AspectJ to configure and otherwise advise these objects, but it can be a useful approach for those using Spring AOP but not yet AspectJ.  This code uses the ability to explicitly create @AspectJ proxies in Spring AOP code, which is also fine.

It's been interesting working a bit with @AspectJ with Spring: you can get good tools support by turning on AJDT while being ableto preserve very natural incremental development in Eclipse. I'm still continuing to use traditional AspectJ syntax for my projects, but like Spring AOP it can be very useful in projects that are starting to adopt AOP:

...
        @Pointcut("execution(music.model.Playable music.model.PlayableDao.find*(..))")
void createPlayable() {
}

@Around("createPlayable()")
public Playable proxyPlayable(ProceedingJoinPoint pjp) throws Throwable {
Playable created = (Playable)pjp.proceed();
return proxyPlayable(created);
}

public Playable proxyPlayable(Playable playable) {
AspectJProxyFactory factory = new AspectJProxyFactory(playable);
factory.addAspect(this);
if (playable instanceof PlayList) {
proxyChildren((PlayList)playable);
}

return (Playable)factory.getProxy();
}

void proxyChildren(PlayList playList) {
for (ListIterator

it = playList.getEntries().listIterator(); it.hasNext();) {
Playable child = it.next();
Playable proxy = proxyPlayable(child);
it.remove();
it.add(proxy);
}
}

September 06, 2006

Glassbox 2.0 automated troubleshooter Goes Beta

I'm pleased to announce that we just shipped the Beta release of the open source Glassbox 2.0 automated troubleshooter. Glassbox 2.0 is a major step forward in simplifying troubleshooting enterprise Java, just deploy it to your app server and it discovers your existing applications and automatically diagnoses problems. Glassbox provides a concise analysis of what’s wrong, focusing on relevant information like excessive queries, or parameters that cause failures.

Glassbox now offers an AJAX Web Client, improved analysis and an automated installer that makes it easy to monitor Java applications in either a standalone or clustered environment.  Under the hood, Glassbox continues to improve its monitoring and analysis to more easily pinpoint problems, Glassbox monitors selectively to allow very low overhead data capture, and we have been working with the AspectJ project to significantly reduce memory and startup time overhead. Glassbox is all open source with our CVS repository now hosted at SourceForge.

Please try it and let us know what you think.

screen shot

We’d enjoy showing you in person, too. We’re presenting Glassbox this fall at:

March 29, 2006

Industry Case Studies at AOSD 2006

Rob Harrop of Interface21 presented how Spring users were following the three phases of AOP adoption ranging from exploration and enforcement through infrastructure aspects and on to core domain uses. Rob highlighted some interesting examples of Spring users at each phase, ranging from reporting on enforcing architectural policy to using AOP to keep database views current. It is interesting that Rob uses IntelliJ with @AspectJ syntax: his AspectJ users are all on Java 5 and in general he sees projects that are still on 1.3 or that are on 5, but not many on 1.4. Ali Duck presented interesting findings from interviewing real world projects that were using AOP, mostly with great results.

Uwe Hohenstein of Siemens presented using AOP to automatically update database statistics to keep good query performance and Michael Mortensen of Hewlett Packard presented how they are using AOP with frameworks. InfraRED presented their approach to application performance management with AOP, now also using AspectJ 5 load-time weaving. Dean Wampler gave a nice presentation on lessons from his Contract4J tool that uses AspectJ to support Design by Contract. I also presented a case study on how we've been using AspectJ for monitoring UI and system events to improve the UI and to support macros at DaimlerChrysler with car diagnostic tools.

Alex Vasseur presented on BEA’s prototype of AOP support in the VM, an initiative I hope to see them continue investing in. Andy Clement showed the latest and greatest AspectJ Development Tools for Eclipse. Not only do they keep improving but I actually think AJDT is catching up to JDT, which is no mean feat. Those of us who use AJDT often complain about what’s not there, but it’s really impressive how much IS now working.

Demos at AOSD 2006

There were seventeen demos at AOSD, most of which were quite interesting. I was only sorry I couldn’t attend more of them! I presented the Glassbox Troubleshooter which uses AspectJ for Java application troubleshooting and the Glassbox Inspector data collector and had some good discussions with attendees about how it works and future directions.

I enjoyed seeing AO4BPEL which uses AOP for Business Process Mamanagement, by extending IBM’s BPEL engine BPWS4J to allow aspects to interact with business processes (e.g., adding promotions or performance monitoring across processes) and also to provide middleware aspects (like reliability and security) for SOAP messaging. I also was interested in a talk about Phoenix.NET, a research project for AOP on .NET, along with a discussion of how Microsoft is also concerned about a number of facets of AOP, in many ways relating to losing control. This probably sounds familiar.

March 27, 2006

Enterprise AOP Tutorial at AOSD

The week started off well as Dean Wampler and I presented a full day tutorial on Enterprise AOP to about 20 attendees. It was a great audience, mostly of experienced developers and architects who wanted to take their AOP and AspectJ knowledge to the next level. We taught the concepts of AOP and how to apply them in the morning. Then in the afternoon, everyone did well on the hands on exercises, which always test you after things sounded so straightforward in the lecture.

This followed a well attended AOP talk I gave at the Software Development West conference the week before. In both cases, I'm seeing more experience and knowledge among developers who come to talks about AOP, as well as interest in pragmatic questions about the next steps in adoption.

AOSD 2006 Conference Overview

I just returned from AOSD 2006 in Bonn, Germany with over 250 people in attendence even with TheServerSide symposium and EclipseCon happening the same week. There was good representation from architects and developers who came to share their experiences and learn more, as well as participation from tool developers and a large contingent of researchers and grad students. It's great to see the caliber of students who are excited about AOP and recent graduates who are using and promoting AOP at their new companies. 

Next year the conference is in Vancouver, and I'm responsible for publicity. I'd love to hear your ideas for how we can reach out to more of those interested in aspects. Some of this year's highlights follow as additional blog entries.

March 17, 2006

My New AOP@Work Article: Next steps with aspects

I'm pleased to announce the publication of a new article I wrote called "Next Steps with Aspects: After Learning Advice" as part of the AOP@Work article series on IBM Developerworks. The article gives advice for effective applications, integration, and adoption at each of these four stages of using aspects:

  • Learning and experimenting. I show examples of exception logging, performance tracing, and enforcement aspects and a technique to reuse library aspects.
  • Solving real problems. I discuss how to integrate AOP with tools and managing dependencies,  show how to implement license enforcement and error handling and how to extend library aspects using Glassbox as an example.
  • Integrating aspects into core development.  I look at using aspects for fine-grained authorization, keeping persistent relationships in sync, and for monitoring and recording macros in a UI. I also discuss some more advanced techniques such as exposing business relationships.
  • Sharing with others. This section looks at how to create reusable aspects effectively, looking at pointcut interfaces and using aspects to allow for feature variations.

I'd welcome your input on the article, and would love to hear how you are using aspects.

February 28, 2006

Glassbox Troubleshooter & Inspector 1.0 Released

We just announced the 1.0 Release of the Glassbox Troubleshooter product and the Glassbox Inspector project. The Glassbox Troubleshooter builds on the Glassbox Inspector’s AspectJ and JMX monitoring technology. The Troubleshooter adds root cause analysis and a focused GUI client that dramatically simplifies the process of Java troubleshooting. Our goal is to winnow down most of the common performance and reliability problems in enterprise development, providing a concise summary of what is wrong with supporting evidence to make it clear. The product is free to try, and we’d love your feedback! I see the product as especially helpful for finding problems in production and when integration or system testing code. See the screen shot below for an example of the kind of information it provides. You can also learn more at http://www.glassbox.com

Just last week, I had a great experience working with a manged service provider where they were able to download, install, run the product and get valuable information about their application in a QA environment in less than half a day (and half of the time was spent figuring out how to uninstall another monitoring product!).

p.s. Stay tuned for more about designing the next releases.

Glassboxtroubleshooter_1

January 05, 2006

First the Top Ten Then...

I was pleased to see today that on Eclipse’s downloads page it now lists the top 10 downloads from eclipse.org. And AJDT (the AspectJ plugin for Eclipse) was #10 on the list. That’s great news to see! J  It's another nice validation of the traction AOP is getting.

I’d be interested in knowing what the download rates for AspectJ and AJDT are these days… I think it was 10,000-20,000 per month late last year. Moreover, when I looked recently, the core Spring framework is at at least 50,000 downloads per month these days. And as Adrian, Alex, Jonas and others have blogged Spring 2.0 promises to be a big step forward in the maturity of aspects, making incremental adoption easier than ever. Of course JBoss 4 is building their EJB 3 container on AOP too.

I had a great time last fall speaking at places like Software Development West, OOPSLA, and the Colorado Software Summit. At the Colorado Software Summit, Roberto Chinnici of Sun talked about including proxy-based AOP as a promising future for the J2EE specification, which would be great validation.

I think this year we are going to see more developers using AOP as an effective tool to get work done, and they are going to find it easier to pick up prebuilt tools that can be extended with AOP. For example, our Glassbox Inspector open source project lets you monitor performance and failures in Java applications. And if you want to monitor something new, you can write an aspect in a few lines of code (or XML) and extend the framework. You can even package this as a jar and just deploy it with the Glassbox Inspector and with load-time weaving it just works. And I believe this is going to be the year where we will see more and more interesting aspect libraries and interesting ways that plain old open source projects start capitalizing on aspects for extensibility (e.g., publishing pointcuts instead of old-school callback hooks, or adding in optional features through aspects).

p.s. I'm also glad that Jonas is moving to the Bay Area this month. Maybe we can do some hacking together and integrate some of the optimizations I've made to load-time weaving into AspectJ's CVS HEAD. In the meanwhile, stay tuned for an experimental release if you'd like to test out a much lower memory version of load-time weaving ...

December 20, 2005

A Hidden Treasure: Weaving with References to Missing Third Party Libraries, and Weaving on Installation

One of the nice little features of AspectJ 5 that isn't documented (at least no where that I can find) is the valuable new Xlint warning cantFindType;  This solves a major problem AspectJ users have faced until now when working with third party libraries. Previously when weaving into a jar file, you would to have to track down all the dependent jars  of that library and put them on the classpath so AspectJ could resolve all the types. This could be quite difficult when weaving into libraries like Spring or Hibernate, where there are many adapter classes that support a large variety of, e.g., cache managers.  It's also a problem in load-time weaving. In that context, a smaller example I ran into was when weaving into the Struts jar, where Action that has a type reference to a legacy data source that is not normally shipped with Struts. The Java code will not even invoke the method that refers to that data source, but by default AspectJ tries to weave everything at once and when it fails to resolve it, it will error out.

However, now you can fix the problem, telling AspectJ to simply ignore (or warn you) for missing types.  Simply create an Xlint.properties file and either copy one (like this one from the Glassbox Inspector project) or create a blank one. You can then add or edit two entries:

cantFindType = ignore

cantFindTypeAffectingJPMatch = warning

You can then use the Xlint properties file to weave with AJDT in Eclipse, with an ant task, with maven or the ajc compiler (or even a smaller command-line weaving utility I talk about below). You can also use it by adding a reference to it in the META-INF/aop.xml file for load-time weaving.

Put together, this allows you to weave and run with incomplete dependencies. I was reminded of how nice this feature was tonight when I ran into a problem weaving into a project jar because there is another configuration of the code that uses a 3rd party jar. On this project, we are doing a command line weave on installation for a GUI application, to allow us to minimize the time for weaving and to minimize change sizes. whereby an aspect was choking on a jar that had a reference to an unused 3rd party library in one, even with smaller configurations of your own application. So we don't want to include all these optional libraries in our distribution! Instead, I was able to just create an Xlint.properties file and fix the problem. By the way, I also created a tiny 20k contribution to support command-line weaving with just the aspectjweaver.jar instead of pulling along a few megabytes of additional AspectJ compiler code. If you want to use that please let me know and I can upload a few small updates I had to make and even a jar. Hopefully we'll get this integrated into AspectJ 1.5.1

It is worth noting that if you do this, you should make sure you won't be running the woven jar in an environment where the missing dependency is present... I think this is so useful that it is probably worth making the case of missing types a warning rather than an error. I also think it's a little surprising that it is an Xlint property. But I'm sure glad the feature is available!

before() : christmas() { release(); }

This morning some great news arrived in my in box. AspectJ 5 final shipped today. It's been over a year in the making and I'm really excited we've hit this milestone.


I for one want to say thank you to the AspectJ committers for working so long and hard to release AspectJ 5! Adding generics support to AspectJ was certainly a major undertaking and it’s great that the end result has been so stable and, in fact, greatly improved even for older Java VM’s. This is a real milestone in the history of aspect-oriented programming and a great Christmas present for all of us.


If you are using AspectJ, I'd encourage you to download it and the Eclipse tools . If you aren't yet using AspectJ, this is a great time to try it and see for yourself what aspects can do!

October 01, 2005

Load-Time Weaving for WebSphere Available

Ramnivas and I are working on an interesting project applying AspectJ to do fine-grained security in a WebSphere 6.0 environment. We wanted to use load-time weaving to let us build modules separately in the RAD IDE and then weave our aspects at load-time. We also wanted to mock the EJB container behavior for integration tests. However, WebSphere 6.0 uses IBM's Java 1.4 VM that doesn't support any Java agents for weaving.

However, I was able to track down the details of the WebSphere ClassLoader Plug In API (the hard part) and write a simple Plug In that lets you do load-time weaving with AspectJ 5 for application classes in WebSphere using its embedded Java 1.4 VM. For older WebSphere implementations, this should also work on their Java 1.3 VM's. You can download the jar and also read about how to use it in more detail at the AspectJ bugzilla report in which I contributed it (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=111027).

September 14, 2005

Article Announcing Glassbox Inspector: AOP Performance Management

I'm pleased to say that IBM Developerworks has just published part one of my two part article, Performance monitoring with AspectJ, Part 1: A look inside the Glassbox Inspector with AspectJ and JMX. The article shows the design of a larger-scale system with AOP, based on real world experience.

The Glassbox Inspector project makes it much easier to troubleshoot enterprise Java applications in production. We announced the open source project's first release yesterday, and would love your input and contributions to the project!

The Glassbox Inspector combines AspectJ and JMX for a flexible, modular approach to monitoring performance for enterprise systems. It provides correlated information to allow you to identify specific problems, but with low enough overhead to be used in production environments. It lets you capture statistics such as total counts, total time, and worst-case performance for requests, and will also let let you drill down into that information for database calls within a request.

May 28, 2005

Virtual Mocking... with jMock

Testing is one of the areas for applying aspects that I have been excited about for the longest time. On some of the first consulting projects where I used AOP, I used a very simple form of "virtual mock objects," e.g.,

public class TestAccount {
...

    pointcut inFailureTest() : cflow(execution(* TestAccount.test*Failure());

    before() : call(* Statement+.*(..) throws SQLException) && inFailureTest() {
        throw new SQLException("account access failed");
    }
}

This was the same kind of testing that Nicholas Lesiecki wrote about in "Test flexibly with AspectJ and mock objects." Subsequently, when I was working on the aTrack project, I created a small framework to simplify this kind of testing for virtual mock objects. First I and subsequently Nick also presented about this approach and framework several times (e.g., see the slides).

Around the same time I created the virtual mocking framework in the aTrack library, Chad Woolley created his own virtual mocking framework, and Monk and Hall wrote about a simple framework. The biggest difference between the aTrack approach and the other two was that aTrack used inter-type declarations to allow defining mock behavior on the actual collaborating objects (and monitoring behavior of tested objects), e.g., Customer.setThrowable(new RuntimeException()). This also meant that when using it, tools like today's AJDT can statically determine a fairly limited scope for virtual mocking (at least unless you use it to replace many classes in your system), whereas the other approaches tended to advise every join point in the system with a runtime check. In future, I expect commercial AOP will be better able to statically determine where pointcuts can match.

Part of this story, though, is the parallel development of mock objects as a technology. Mocks are still fairly new, and most of the people who use them practice agile methodologies. Whenever I presented "Testing With Virtual Mock Objects," I found that people who actual had tried to use mocks and those who used TDD were excited. Many others were excited by ideas that were really based on the idea of mocks.

One important development in mock objects has been the jMock project. It uses dynamic proxies and features a concise and readable syntax for writing mock tests.

As I've thought about the existing implementations of virtual mocking I realized that they all suffered from trying to reinvent a good, clean API for mocks. And with jMock being a big improvement on past APIs, it would be a lot better to extend jMock to support virtual mocks...

My thinking is that there are really five interesting types of mocking:

  1. direct collaborators with public interfaces (classic mocks)
  2. direct collaborators with non-OO interfaces (singletons, static methods, private methods, etc.)
  3. indirect collaborators (unit tests of closely coupled classes and integration tests)
  4. aspects
  5. monitoring the object under test

I think the right unit of abstraction for a virtual mock is ... the pointcut and not a type. (Aside: this is related to my contention that it is better to define and configure loggers by pointcut rather than type). With this in mind, I wrote some proof of concept code that successfully extended jMock with the concept of a virtual mock in two different ways. I've been calling this ajMock...

  • Dynamic
    class TestInitialization extends VirtualMockObjectTestCase {
    ...
        public void testInitialization() {
            VirtualMock mockStart = virtualDynamicMock("call(* startUp()) && target("+ApplicationLifecycleAware.class.getName()+")");
            mockStart.expects(once()).will(doProceed());
            ...

    Static
    class TestInitialization extends VirtualMockObjectTestCase {
    ...
        public void testInitialization() {
            VirtualMock mockStart = virtualMock(StartInitialization.aspectOf());
            mockStart.expects(once()).will(doProceed());
            ...
        }

      private static aspect StartInitialization extends VirtualMockAspect {
            public pointcut mockPoint() : call(* startUp()) && target(ApplicationLifecycleAware);
    ...

It was relatively straightforward to integrate these extensions into jMock (albeit incompletely and as more of a test than a complete framework). One really nice thing is that you can use the normal jMock APIs and should be able to mix and match proxy and even CGLIB proxy mocks with virtual mocks. There are a few areas where jMock assumes that mocks have proxies, and where virtual mocks can do differently.

I've been using this code to test my performance monitoring code and was pleased that it has helped me (mostly with integration tests). I started off with the dynamic approach, extending the new AspectJ 5 reflection library. However, I found that tools issues made this unpalatable. The small issue was the annoyance of runtime checking of pointcuts and having no tools information about them. I adopted the idiom of using Type.class.getName() when writing the mock strings to get compile-time checking rather than using runtime checks to help a little with that. The big issue that led me to change to the static approach was the impact on tools support everywhere else. The dynamic version had to advise almost every join point in the system with dynamic tests for whether to mock it. This resulted in a painful experience when in stepping a debugger, it added too much clutter to gutter tips, cross-reference views, etc. for advice (since everything was advised), and even made stack traces harder to read. The static version seems like a pretty reasonable compromise.

I think dynamic virtual mocks are a great candidate for load-time weaving: you could apply them to a limited scope for a given test case and not impact tools support when editing or debugging outside the scope. The part I haven't yet designed is how to effectively integrate load-time weaving into Eclipse's jUnit launcher.

These type of virtual mocks can be a great complement to, e.g., jMock.

  • It makes it a snap to mock things that are traditionally hard to mock (static, final, or private methods). (case 2 above: non-OO interfaces)
  • It also makes it a snap to track behavior of objects under test, without in any way mocking them. (case 5 above)
  • Pointcuts are great for expressing systematic mock behavior where jMock method patterns names aren't robust enough (I see their matchers as just a baby step towards the power of pointcuts). E.g., what if I want to specify a null return value for any method returning an Object or subclass, and false for any returning a boolean? What if I want to throw SQLExceptions from any database call? (adding to the value that jMock provides in case 1)
  • Pointcuts also make it really natural to integration test an isolated subsystem. Often you want to plug in dummy behavior that gets invoked a few calls down. With ajMock, you can specify this easily. In integration tests, it is sometimes a big win to use an object's own behavior as a starting point, and just to change it a bit to make your test case work (rather than dummying out all the interactions).
  • I think this approach can be extended to achieve many of the goals of, e.g., aUnit, but in a way that lets you integrate with other mocks. I think you'd need a different kind of virtual mock that let you generate (stub) pointcuts, rather than just matching them in the normal flow. However, the jMock-style API seems to be a useful one to allow testing pointcuts meaningfully, through representative examples. This is a rich area for further exploration. (case 4 above)

I'd be interested in thoughts from others about this? For jMock users, where would you see this as a useful complement. Obviously I need to flesh out the design and explain more of it. I also think there's a lot of room to extend the simple virtual mocks with common idioms (e.g., plugging in a regular jMock of a class that is invoked indirectly, mocking an assembly of objects).

May 05, 2005

Load Time Weaving with AspectJ 5

I've been working with the new AspectJ 5 Load Time Weaving implementation that Alex and Jonas have created, and am really excited at the opportunity this technology brings. I have tried it with the Sun Java 1.5 VM and the JRockIt 1.5 VM too (*).  I'm working on creating application monitoring tools, so it's really natural to weave into the whole VM that a container is running in. Using LTW, we are already able to track the operations of any deployed application inside Tomcat 5.5.x, which is very cool!

I have used build-time weaving with third party libraries in past and found it very painful because you needed to either find any libraries that these libraries used (e.g., to weave into Spring.jar you need all the various O/R tools, JMX, and a bevy of obscure cache management libraries). With load-time weaving, you only weave the classes that are loaded, so you only need the libraries that are referenced by these. That's a big improvement.

LTW is still "pre-alpha", so naturally I faced a few gotchas (I was working off a branch in CVS and they just merged it into head in the last couple of days). Thanks to Alex for helping me with these and to Andy Clement for bearing with my questions about mixing 1.5.0 M2 output with the fairly old branch code!

One of the things I'd like to change is the need to specify all the aspects that are woven in an aop.xml file. Alex and I traded thoughts on this and my idea of an aspectpath like mechanism on bugzilla (where else)? I would really like to have some way to generate the list of aspects, rather than having to maintain it by hand, hopefully stored separately (in something like META-INF/aspects.mf).

I also ran into an interesting problem when I was weaving into a MySQL JDBC driver's debug version:

error at com\mysql\jdbc\NonRegisteringDriver.java::0 class 'com.mysql.jdbc.NonRegisteringDriver' is already woven and has not been built with -Xreweavable

That's right. This error means that the code was already woven with AspectJ! The MYSQL debug JDBC driver includes a tracing aspect: it's great to see this kind of increased use & popularity of AspectJ. For the curious, you can see the source code for this aspect in the source distribution of the Mysql Connector/J. In this case, the normal, non-debug, library doesn't use AspectJ so it's not a major problem.  There is already consensus that reweavable should be the default mode in future. I even question why there's an option to not allow reweaving. (**)

Now the next challenges I see on this front are:

  • Shipping 1.5.0
  • Optimizing the weaving performance, especially for simple cases. I'd really like to see AspectJ's load-time weaving fairly close in performance to hand-written bytecode manipulation. Not necessarily the same, but at least close enough.
  • Support for older VMs (which I know Alex and Jonas are working on)

Ron

(*) I found JRockIt to be about twice as fast, but to not be quite as good at debugging my aspects inside Eclipse. Apparently BEA has a patch for Eclipse debugging but is only providing it to paying customers...

(**) Amusingly, I only discovered this because I had an error in the first zip distribution I downloaded so I could only extract the debug jar file.

April 03, 2005

AOSD 2005 BOFs

I attended four BOFs at AOSD 2005.

Testing Aspects. The practitioners at the session felt that TDD and testing aspects was quite natural and worked well. Many of us use JUnit to achieve this. Common themes include:

  • Making the aspects thin, so helper classes can be easily unit tested (e.g., to unit test advice).
  • Creating "test driver" static inner classes that exercise the right functionality. Note: these classes typically are like mocks in that they have no real behavior and track interactions, but are unlike mocks in that they have some control flow to expose join points. These exposed join points test cases that should match and shouldn't match to test pointcut definitions. Some edge cases that may worth trying include handling subclasses, superclasses, even inter-type declarations (aka introductions).
  • Making aspects abstract, so test cases can include a static inner aspect that concretizes the pointcuts to make them apply to the test driver classes.
  • Testing tightly coupled aspects with their collaborators as a unit.
  • Use IoC to configure aspects (just like with objects).
  • Adrian noted that there are a whole bunch of little techniques for unit testing aspects, like using static inner aspects inside a TestCase and delegating to logic methods from inside advice. He also discussed the aUnit project, which he "announced" previously as a way of recruiting someone to build it. Russell Miles is currently working on it. I think it's really a kind of mock framework for simulating join points. This kind of framework can help test advice, and could facilitate testing without having to place code in multiple packages. However, there's a danger in not testing pointcuts by using the same specification for the pointcut as for the test.
    Adrian also noted that the AspectJ library will be a chance to show TDD for AOP.
  • Jonas noted that because AspectWerkz has named advice, it was easy to unit test it and mock it out also (note that you can use virtual mocks with adviceexecution to mock out AspectJ advice...).
  • There would be value in integrating use of aspects for testing with a jMock-style mocking framework.

Aspect libraries. There was a great deal of interest in creating reusable aspect libraries. I will post more about this BOF in a separate posting on the topic.

AspectJ 5. This started with Adrian demoing some of the cool new AspectJ 5 features. We discussed support for type patterns, and composability for type patterns and signatures. With the complexity of Java 5 generics, and the complexity of pointcuts for annotations, I think there needs to be an ability to name type patterns and also a verbose functional syntax for composing type patterns and signatures... We also support for type pattern - (Ramnivas demo'd a case where it would have helped).

Perobject BOF. This BOF was organized by Eric Bodden and Nicholas Lesiecki, to explore how to allow to use/extend AspectJ to associate aspects with instances.

One great outcome of this was an agreement in principle to implement dynamically enabled and disabled aspects at runtime for AspectJ. The basic idea is to allow an aspect to implement:

public interface Activation {
    boolean isAdviceEnabled();
    void setAdviceEnabled(boolean isEnabled);
}

For any aspect that implements this, the compiler will generate the default implementation (like an ITD), and will guard all the advice for that aspect with a test for ...  && adviceEnabled. This test should be like a normal if test on a pointcut (i.e., be inlined in the method body), but it would reference the concrete aspect instance(s) that might match.

Adrian's recently discovered pattern of using static inner aspects makes it feasible to instantiate a perthis aspect in a static context. This can help in some cases. Eric and Nick posted more thoughts on this to aspectj-users and Adrian showed an example which I'm sure he'll be blogging soon.

March 21, 2005

AOSD 2005 Industry Track: Fri Mar. 18

The industry track concluded with Olaf Spinczyk's presentation on Aspect C++. It's been over ten years since I've done any real C++ development, so I attended mostly to get a sense of how adoption of the technology is going. I sometimes get asked about using AOP for C++ when people attend classes and tutorials I give, so it's nice to know more about this implementation.

AspectC++ has had 500 downloads since they did a release on Feb. 10. Nokia has used it to weave into production Symbian OS code (cool). Siemens and Samsung are trying it out too. There are still some limitations (C++ is a beast of complexity, so adding AOP is taking time): they are focused on source-code weaving (cf. to cfront).

Sam Pullara was our "closer" and he did a great job presenting on VM-level AOP. He compared different stages of weaving and analyzed how they work.

Some interesting problems in load-time weaving he identified include:

  • serialization of woven classes might break (due to incompatible formats if not woven on both client & server)
  • reflection in woven classes might break (e.g., if the application iterates over all methods or fields, even synthetic ones)
  • there are possible incompatibilities when there's more than one agent/weaver (e.g., will AspectJ aspects weave into Hibernate's dynamically generated proxies?)
  • the presence of shared system class caches could cause problems for scoping aspects to affect specific classes (I need to learn more about this)
  • stack traces can have synthetic frames (minor inconvenience)
  • increased amount of memory required for analysis
  • increased classload time

Sam also argued for more expressive pointcuts, e.g., exposing BeanInfo information (which would be a better way to match getters and setters and events too).

Adrian has blogged a much more exhaustive summary.

Toplink Persistence Aspects

The next industry track session on Thursday was Object Persistence Aspects by Jim Clark of the Oracle TopLink team. Their first use of aspects was to policy proper application development, notably ensuring that developers didn't modify objects in the shared read-only cache.

They next developed a way to generate AspectJ aspects from their XML mapping files to support lazy loading and also to replace snapshot-based differences with active dirty tracking while operating on persistent objects for "read mostly" applications.

They subsequently built a custom weaver/bytecode instrumentation engine using BCEL to implement this requirement since it could understand their XML (not require a separate step in the build process), and because they optimized the performance significantly compared to the memory and time required for general-purpose AspectJ weaving. I think this is an interesting case of thinking in aspects being a benefit: at least they could prototype the right implementation quickly, even though they ended up hand-tuning an optimization.

They still require a build-time option for deployment to "non-managed" environments (e.g., 2-tier deployments for testing or unusual app servers) where they can't weave at load time.

They are interested in leveraging general-purpose AOP in future, rather than maintaining their own, and see opportunities to provide higher level aspects like session management, query prefetching, and transaction demarcation. The initial feedback has been positive to the new "aspect-based" persistence option. They see aspects as the right answer, and preferably provided by the creator of a persistence framework.

NearInfinity: AOP for Auditing J2EE

Matt Wizeman and Jeff Kunkle of NearInfinity presented a practitioner report describing how they used AspectJ to build a product for enterprise compliance auditing. They note that most projects add auditing as an afterthought and it shows in the poor quality results. One key benefit of separating out the auditing implementation is it allows consistent policy implementation (it takes audit decisions out of the developers' hands).

NearInfinity uses AspectWerkz aspects to collect data, which then gets fed into a pipeline:

collect -> transport -> filter -> transform -> alert -> store

The data is collected into an in-memory database, and the architecture allows filtering data in an XML form at several places.

The system uses load-time weaving with AspectWerkz. They were initially using AspectJ with the Weblogic Aspect System, but converted when the latter wasn't updated for AspectJ 1.2. The AspectJ 1.1 version took 4 minutes on system start up. The AspectWerkz 2.0 version took 50-60 seconds. They are planning to use AspectJ 5 when it picks up comparable LTW support. This lets them weave into several interesting places:

  • generated code (e.g., entity beans)
  • JSPs
  • 3rd party libraries

They also have thin aspects, moving as much code as possible into helper classes. One key thing they emphasized is the importance of aspect robustness: e.g., they hit a bug when beta testing because they were trying to log a null argument to a JNDI lookup in a customer system. The customer system was swallowing the resulting NPE so it was hard to track down. They key lesson is to be paranoid.

As a side note, aspects can also help with this problem. You can write after throwing advice to catch any exceptions thrown out of advice you write, to prevent it from poisioning the underlying system (e.g., log it and swallow it). This is very appropriate for auxiliary aspects, or any part of the system that isn't essential for core operation. Note that you can't quite make this work for around advice: if there's an exception in around advice you don't know if it preempted the proceed call... This is a great candidate for a reusable aspect too.

One key part of their system is tracking SQL data: they have gone through 4 different approaches:

  1. wrapper pattern (on construction)
  2. per instance aspects (but the Weblogic EJB deployer choked on this)
  3. storing a TheadLocal HashMap of statement to state (I believe Nick pointed out that this is probably better done with an InheritableThreadLocal, as Wes has suggested in past).
  4. generating a dynamic proxy to wrap a statement on creation

I had previously discussed with them how one might weave into the JDBC driver code to address this problem, while still allowing per-application data (e.g., tracking information by application). I am optimistic about this approach for situations where it's reasonable to have your aspect affect all the applications in a container. In many cases, it's quite reasonable to deploy some applications to different containers, too.

Another interesting implementation issue is how to audit requests that don't reach user servlets and how to map to the right application (e.g., 404 errors or static resources). This is very important for auditing (e.g., to monitor sql injection attempts).

So they use a Servlet filter, which has the right hooks defined portably. Sam Pullara suggested advising an empty filter. It's an interesting question if one can portably add a filter at runtime, or achieve the equivalent affect with just advice (i.e., avoiding the need to configure applications' web.xml files).

In future, they plan to expand coverage to include system commands, socket I/O, file manipulation, exception handling and 3rd party search engine APIs, Web frameworks and persistence frameworks.

JBoss: Implementing Middleware Using AOP

Bill Burke presented how JBoss is using AOP to provide transparent middleware. It was interesting to hear JBoss's new message that most of their users want to use aspects but not program them. They are promoting the message of annotations for ease of use and aspects for ease of extension. At the same time, JBoss is making extensive use of aspects: it's the basis of their EJB3 container (log, security, transactions, locks, remoting, etc.). It's great to have the fastest growing J2EE container build the core of their new platform on AOP!

Their transparent replicated JBoss Cache is a very cool application of aspects. Interestingly, they are using aspects to build it for product line variation. It lets them provide pluggable eviction, persistence, replication, and transactionality support without tangling the core cache code. They've also been able to use "advice stacks" to provide common elements for different J2EE components (SSB, SLSB, MDB, ...), while capture the variations. The advice stack concept in JBoss AOP is a kind of composite pattern; it would be interesting to better understand how it compares to AspectJ-style aspects.

They are also going to provide a capability to use XML to define or override programmatic @Annotations, and they allow changing annotations on a per instance basis. JBoss AOP lets one configure metadata per invocation, thread, class, vm, or cluster.

They see it as important to be able to hot deploy aspects (binding or unbinding at runtime, e.g., to add metrics at runtime). They also support per-instance advice. Bill told me that the weaver adds if tests, so this should be performant even for call join points.

Futures for JBoss AOP include a DI/IoC container in the manager environemnt and potentially being able to defer initialization until all dependencies are available.

Application Security Aspects

I then presented a session on Application Security Aspects. I showed examples of how AOP can implement non-trivial application security requirements like Web application authentication, role-based and data-driven authorization, enable defense in depth (checking rules at each tier) and ensuring proper audit trails for any security decisions. A nice property of the AOP solution is it provides traceability from the design to the implementation and more understandability to end users.

AOP can also add value to data access security checks; I showed an example of checking the security context to restrict queries using TopLink APIs (which is much easier than parsing and modifying SQL strings!). It can also add value to filtering out UI entries the user isn't entitled to see (fields or commands/buttons/links). The latter problem is often hard because of difficulty integrating with UI languages like JSP and because it's hard to know what the boundaries of a UI control/section are in typical markup (unless developers use custom tags that define controls or add some kind of XML annotation - marker tags). Overall, AOP is a great way to implement a number of typically scattered security designs, and is a big help on implementing others, though there is more work required to make it easy. All of this is ripe for reusable aspects that do a lot more for you than checking JAAS permissions based on an annotation!

The slides for this presentation are available online at http://www.aosd.net/2005/archive/Security_Aspects.ppt

AOP for Spring

The industry track on Thursday started with Rob Harrop and Adrian Colyer presenting AOP for the Spring framework. Spring integrates coarse-grained proxy-based AOP into the core framework and uses it for transaction management, security, remoting, performance monitoring, tracing and logging, and framework internals. Rob noted that many organizations adopt Spring that aren't aware of AOP. Subsequently it becomes validation, e.g., if a Spring consultant suggests using AOP, they can point out the customer has been using AOP in Spring for 6 months to assuage any fears.

The second part of the talk covered integration of AspectJ with Spring for more advanced uses. Adrian has blogged about this in past: both configuring AspectJ with Spring and extending Spring with fine-grained aspects in AspectJ. Adrian showed how to separate instantiation from configuration for non-singleton aspects. This used an aspect that has Spring configure a bean after instantiation, using an @Bean annotation. This is useful for beans too, that need context-sensitive instantiation (e.g., setting sessions on DAOs).

The next release of Spring will include an <aspect> xml tag (like <bean>) to make them easy to wire up. And there is significant collaboration to integrate Spring and AspectJ.

AOSD 2005 Industry Track: Wed Mar. 16

The track started with Gregor and Adrian's talk on adopting AOP. This was great and well worth drawing from if you are presenting on this topic in your own organization! The presentation had some great examples from internal use at IBM, notably examples of how it allowed system architects to refine the policy for error handling, which was never possible to vary before and the ability to vary features in product lines. Gregor also raised the great ideas of modularizing quality of service and doing clean room system extensions. The slides from the keynotes and the industry track are being posted at https://www.aosd.net/archive/

The basic approach of building up from exploration and enforcement to auxiliary/infrastructure to core/domain-specific is fairly proven. I find managers and architects who are interested often want to "spike" ahead and get help to prototype a compelling production use case too. I am also seeing more interest in analysis, design, and strategy for longer-term use. I also really like the graphics and their emphasis on being modest in claims, since people often want to hear about silver bullets. In particular, they wrap up classes with a slide about "What makes a good aspect," to emphasize that aspects should be used to solve hard problems modularly, i.e., to improve the design & implementation.

Then Adrian and Jonas talked about what's new in AspectJ 5, covering how AspectJ and AspectWerkz came to merge, and showing examples of how it will support annotations, generics, and other Java 5 language features. The use of type erasure with generics eliminates runtime type information, which makes it harder to use pointcuts. I won't even try to recap this material: take a look at the AspectJ 5 developers handbook for more detail.

This talk and Jonas' talk on the road to AspectWerkz 2, also covered the origins and implementation for AspectJ 5 of classloader-aware load time weaving (aka LTW) and @AspectJ syntax.

Classloader-aware LTW means you can deploy an aspect to a classloader and it will weave into only classes loaded from that loader (or below in the hierarchy). It will be using Java 5 agents and JRocket agents. I spoke to Alex and Jonas subsequently and think we'll see options for deployable Java 1.3 and 1.4 LTW support that works with containers, too. The ability to use aop.xml to control weaving and to apply abstract aspects are great too.

Jonas also showed the @AspectJ annotation, which brings the Aspectwerkz annotation-defined syntax to AspectJ (I had suggested this approach to Jonas and it became the more popular syntax for AspectWerkz users). I also expect to benefit from using it on a project for a customer where everyone uses IntelliJ IDEA.

Jonas also presented the work they did on runtime weaving with HotSwap, programmable per-instance deployment, the extensible aspect container, including a very high performance pure proxy-based AOP implementation, and on the AW Bench AOP performance benchmark.

Sam Pullara asked a great question about how AspectJ's annotation support will work with apt. Adrian noted that you can run apt first, then weave the result with AspectJ. However, it's a good question, because there won't be support for processing the results of declare annotation with apt. I also wonder how Eclipse will provide good IDE support for apt tools; I see the Pollinate project was discussing how Eclipse should work with apt; hopefully they will lift their consideration from making it work to first-class tools support...

Nicholas Lesiecki presented some great information from his team's experiences adopting AOP at VMS:

They first added support for logging initial exceptions for certain popular Web frameworks that don't do exception chaining (e.g., Apache JSTL). This was quite helpful, although allowing after advice on handler join points would have made it much better. I think AspectJ needs to capture additional bytecode metadata to capture enough information to allow weaving properly in the case that the original source was compiled by an AspectJ compiler.

Nick emphasized the importance of the design pattern whereby aspects uses helper class(es) to do the heavy lifting. This makes it easier to inject alternatives, ease tools use, and aids in testing.

They next implemented a basket repricing aspect for e-commerce. This aspect affects 12 points in 3 classes: they have significantly refactored it 4 times, with a 5th underway at the time (based on a new marketing requirement). This is a key benefit of aspects: they would have never felt free to refactor this way in a tangled implementation.

Nick also presented some great aspects that work with Hibernate. They auto-register hibernate objects with open sessions and most impressively, built a fairly generic aspect to implement relationship management (like CMR) for Hibernate. This affects 93 places in their application. It tok took 2 weeks to write. This prompted Merrick Schincariol of Oracle to question whether it was worth it for a single project: I think it depends on how much time it saved in fixing bugs, although maybe a simple enforcement aspect would help a lot. I think an aspect like this that is tightly coupled to a framework is most valuable when reused and is best supplied by the framework itself.

The team adopted AOP fairly easily and quickly: it took 6 months of project time in which 5-10% of development effort was devoted to learning it. By contrast, the project also converted to Tapestry and Hibernate and both were stop the world adoptions. However, aspects were often suspected as a red herring cause of other problems. My code doesn't work? Blame AOP!

Ramnivas gave a great presentation of annotations and AOP, and Mik gave a great presentation comparing the leading AOP tools. Both of them have written two part articles in the http://www-128.ibm.com/developerworks/views/java/libraryview.jsp?search_by=AOP@work: series on IBM Developerworks. I'd encourage you to read them for more details!

AOSD Keynotes and Panel

There were two great keynotes and a fun panel at AOSD this Year. I won't try to capture all the details (Adrian did a great job for both keynotes)... some things I really liked were:

Grady Booch kicked off the conference with his keynote. It was nice to hear Grady say "I don't think aspects are a fad, I think there's something deep and profound about them," and to exhort us to apply AOP to the architectural level. His handbook on software architecture is a very exciting project: who else could get the access needed to document the real architectures of large-scale projects? He also noted that often the architecture is more about key design patterns than functional decomposition into boxes (he noted this in the case of Eclipse).

Grady emphasized the complexity of software environments (there are so many languages, platforms and tools). I think this is actually one of the best things about AOP: it provides a general-purpose tool to address problems instead of gobs of special-case tools (code generators, tools, plugins, interceptor frameworks, annotation processors, XSLT, code checkers, preprocessors, etc., etc.). I ag

David Thomas (OTI founder, long time object pioneer, and founder of Bedarra) presented some great lessons for the aspect-oriented community. Programmers are suffering from "Too Much Stuff" (TMS), and he is positive about projects like Spring that simplify enterprise Java. He sees innovation happening at the edge, and expects today's J2EE and .NET will become legacy systems in 15 years. Bedarra are focusing on solutions for developing in 2010, with support for high performance teams that pair domain experts with developers. Dave is always a great antidote to hype (e.g., who has time for UML), and encouraged us to get smart teams out there to help organizations refactor today's bloated OOP into clean AOP.

He really had some great lessons for AOP providers, based on his experience doing this for objects:

  • listen to the customer problem and relate the new technology to how it solves it, using an example that
  • address the critic's concerns up front, to sell the importance of productivity
  • position yourself between on the "inside" of the change advocate (visionary/early adopter) and the rest of the customer organization

David also lamented the state of computer education today:  not requiring learning functional programming or constraint-based programming, just Java (or just theory with no coding!). He emphasized the importance of good background and an ability to reason abstractly from first principles. He also advocated diverse view points, Brian Smith's ideas, and separation of concerns. And I appreciated Dave's reference to Worse is Better.

The panel discussion discussed a range of adoption issues. It was great to hear from Dion that 33% of TheServerSide Symposium attendees are using or planning to use AOP in production (think of all that arm waving Hani!). Bill Burke raised the question of how to explain AOP in a few sentences to new developers (I like Adrian's buzzword-free explanation...). I think Rob Harrop noted the importance of knowing what's already out there, not replacing existing features (e.g., don't give examples that a Servlet filter can replace). Another interesting point that came up was who should write broadly crosscutting aspects: it's the people who can write frameworks!

Software Development West 2005 Enterprise AOP Tutorial

Ramnivas and I started last week by presenting a full day hands on tutorial on enterprise AOP at Software Development West. We were really impressed by the level of knowledge and smart questions and success that many attendees had! It's a big change from the early days of AOP. We also were impressed that about 30 people came and stayed for a full day of lecture and hands on development. Thanks to all the attendees!

Hieu Nguyen also asked an interesting question: is advice on method-execution of a synchronized method itself included in the synchronized block? I had to think a bit before I guessed the answer would be yes (which I then verified by writing a test case). Of course, if you wrap the method body with a synchronized(this) { ... } block, then the advice is outside the synchronized block.

Given the desire to keep method synchronization attributes in tact, this is really the only way it can be (i.e., you can consider synchronization to occur between call and execution), but it would present an interesting challenge when/if AspectJ gets a sychronized(...) pointcut descriptor.

AOSD 2005 Overview

I have just returned from the 2005 edition of the Aspect-Oriented Software Development conference in Chicago last week. There were 255 registered participants, almost 25% of which were from industry. It was great to see so many people from industry who are using AOP in a serious way, as well as researchers and students who are creating the next wave of aspect innovation.

I flew in on a red eye Tuesday morning and presented a half-day Enterprise AOP Tutorial with Nicholas Lesiecki to kick off AOSD. The audience who came to our tutorial was a microcosm for the AOSD 2005 industry track attendees as a whole: a good mix of forward looking developers and managers and researchers who are interested in real world applications of AOSD.

It was a busy week that featured a great combination of:

  • Presentations about new technology developments and information on using AOP on real projects
  • Informal conversations with AOP technology developers, leaders in applying AOP, and researchers
  • BOFs on AspectJ language design and reusable libraries
  • Keynotes, tutorials, demos, exhibitions, and workshops

As an industry chair, I attended all the industry track, so this year I wasn't able to hear the research papers nor the demos, though some sounded very interesting. The following entries will give highlights from some of the events I attended.

Thanks again to all the speakers, industry participants, researchers, and students who came and made AOSD such a great event. Thanks to all the sponsors and to Mira, Tzilla, Ramnivas, Gregor, Brian and the rest of the organizing committee who were instrumental in making the conference go well and making the first industry track go well. Good luck to the organizing comittee for next year, especially the industry chairs: Alex Vasseur and Matt Chapman!

See also Adrian's great blog entries for more in depth coverage (starting at http://www.aspectprogrammer.org/blogs/adrian/2005/03/aosd_2005_confe.html) and Dion's news posting at TheServerSide.