This issue has plagued my update sites for years and I finally found a workaround.

The problem: you create an Eclipse update site, you add a category, then you add a feature under the category. You build the update site (using either Build or Build All). Sometimes, when you try to install the features from the update site with "Install New Software..." you don't see your category and you need to uncheck "Group items by category". Even if you try to rebuild everything, delete the artifacts.jar and content.jar, you still cannot see your category. Annoying isn't it?

The solution: remove the category and the features from your update site. Save. Add them again. Build All. You need to do this every time you make a change (e.g., update the version number of a feature). This is a silly bug in Eclipse, but it is the only way I could reliably work around the issue.

Root cause: it seems that there is some caching of the site.xml involved and that restarting Eclipse may help, but it never worked for me.

All right, this one isn’t exactly exiting, but while working on Py4J, I encountered the following problem (that had to be solved in Java):

Given a simple name, return a list of fully qualified names accessible at runtime. In other words, implement the auto-import feature available in popular Java IDEs, but at runtime. For example, given List, you should return [java.util.List, java.awt.List].

I had no idea how to do this, so I asked a question on stackoverflow, but to my dismay, the only answers I got were “it can’t be done”. It hurts because one of the commenters was an expert in classloading/reflection issues on StackOverflow. Still, I wanted a best effort solution (not a 100% accurate one) and in a general programming language like Java, “it can’t be done” is not an acceptable answer.

If you want to skip to the code directly, be my guest. Otherwise, read on to learn more about the solution!

My journey thus began and I read a lot about traversing classloaders, inspecting the classpath, looking at the content of a jar file, getting a list of packages available in a classloader, and so on. A simple solution would have been to get a list of all classes available in each classloader. A classloader is an object that is responsible for loading the definition of a class (e.g., when you write import java.util.List;).

Unfortunately, I quickly reached the conclusion that any solution would have to be extensible and that there was no general solution. It is not possible in Java to traverse and inspect all classloaders, even if a classloader will eventually load a class you need. This means that a best-effort solution would just try to load a fully qualified name to know if it is accessible (e.g., try to call Class.forName("java.util.List")). The problem then became: How can I get a list of accessible packages at runtime?

To solve this problem, I created a series of PackageProvider. These are classes responsible for getting a list of accessible packages in specific environment. These are the package providers I implemented:

• Classpath: get a list of all the directories and jars passed in the classpath, inspect these jars/directories and return a list of packages.
• Bootstrap: get a list of all the directories and jars used to load the JVM (e.g., rt.jar containing java.lang, java.util, etc.), inspect these jars/directories and return a list of packages.
• Current classloader: get a list of all available packages in the current classloader (i.e., Package.getPackages(). This could be useful to detect dynamically generated classes created with cglib or Javassist.
• Eclipse: assuming that you are in an Eclipse plug-in and that you can access all plug-ins (i.e., Eclipse-BuddyPolicy: global), get a list of all plug-ins in an Eclipse installation and then, get a list of exported packages for each plug-in.

And that’s it. It’s horribly slow on large Eclipse installations, but it works like a charm. With some caching, the speed becomes tolerable. Other package providers could be created for environments like a web container (e.g., Tomcat, JBoss). Don’t hesitate to look at the code, it’s available under the new BSD license.

Read the rest of this post »

Three more Python tricks (ok, one is about a standard python module) that I always get wrong the first time.

Reading a Web Page

import urllib2
page = urllib2.urlopen('http://www.infobart.com')
content = page.read()  

# content is now a "byte string" (ascii-like) 
# which does not support unicode. 
# Do this to properly encode your content: 
encoding=page.headers['content-type'].split('charset=')[-1] 
ucontent = unicode(content, encoding)  

# Reference: http://stackoverflow.com/questions/1020892/urllib2-read-to-unicode [/cc] 

Sorting Lists

>>> l1 = ['BBB','aaa','ccc','DDD'] 
>>> # Standard sort 
>>> sorted(l1) 
['BBB', 'DDD', 'aaa', 'ccc'] 
>>> 
>>> # Passing a function that will be applied 
>>> # to each value before sorting. Good for 
>>> # standalone function
>>> sorted(l1, key=str.lower)
['aaa', 'BBB', 'ccc', 'DDD'] 
>>> 
>>> # Using a lambda that will return the key 
>>> # for each value: better if you want to invoke 
>>> # an instance method. 
>>> sorted(l1, key=lambda s: s.lower()) 
['aaa', 'BBB', 'ccc', 'DDD'] 
>>> 
>>> # Also works with list.sort() 
>>> l1.sort(key=lambda s:s.lower()) 
>>> 
l1 ['aaa', 'BBB', 'ccc', 'DDD']

String Formatting

b1 = True 
f1 = 7.0/22.0 
i1 = 234 
s1 = 'World!'  

s = 'Hello ' + s1 + str(b1) + str(f1) + str(i1) 
# s = 'Hello World!True0.318181818182234'   

s = 'Hello %s %r %.2f %i' % (s1, b1, f1, i1) 
# s = 'Hello World! True 0.32 234'  

# Preferred way of formatting a string 
s = 'Hello {0} {1} {2:.2} {3}'.format(s1, b1, f1, i1) 
# s = 'Hello World! True 0.32 234' 

Three more Python tricks I miss a lot when I have to program in Java!

Using a Context Manager

from threading import lock;
lock = Lock()
with lock:
    print('I have the lock!')
    print('I still have the lock!')
print('I no longer have the lock')

# no more try/finally:
# try { 
#lock.lock(); 
# } finally { 
#   lock.unlock(); 
# } 
# Can also be used with files or your own objects!

List Comprehension and Filtering

>>> # List comprehension 
>>> l1 = [i for i in xrange(10)] 
>>> l1 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] 
>>> # List transformation 
>>> l2 = [i*2 for i in l1] 
>>> l2 [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] 
>>> # List Filtering 
>>> l3 = [i for i in l2 if i % 3 == 0] 
>>> l3 [0, 6, 12, 18]

Accessing the index in a “for each” loop

my_list = ['a','b','c','d','e']
for (index,letter) in enumerate(my_list):
    print('%i: %s' % (index,letter))

# Will print: 
# 0: a 
# 1: b 
# 2: c 
# 3: d 
# 4: e  

# In Java, you have to either: 
# int i = 0; # for (String s : myList) { 
#     System.out.println(i + " " + s); 
#     i++; # } # # OR # # for (int i = 0; i

I’m launching a new series of short posts on small Python tricks that I slowly discovered and that were not immediately obvious coming from a Java background.

How to call Exception.printStackTrace()?

from traceback import print_exc

if __name__ == '__main__':
    try:
        i = 1 / 0
    except Exception:
        print_exc()

Looping through two collections

namesList = ['Alice','Bob','Carl','David','Earl']
agesList = [25,40,10,5,19]
for (name,age) in zip(namesList,agesList):
    print('%s is %i yrs old' % (name,age))

# For very long lists, izip is better because
# it does not create a list (like range and xrange): 
from itertools import izip

for (name,age) in izip(namesList,agesList):
    print('%s is %i yrs old' % (name,age))

Was the loop interrupted?

namesList = ['Alice','Carl','David','Earl']
for name in namesList:
    if name == 'Bob':
        break
else:
    raise Exception('Bob was not found')
    # Note: else is only executed if no break was executed