Charm4py started development two years ago. Today we are happy to announce the release of version 1.0. This marks an important milestone, containing many improvements that simplify how Charm4py programs are written. The documentation has been heavily updated, and contains an introduction with examples and a tutorial that should help get up to speed.
Some of the highlights of the release are:
Coroutines: threaded entry methods are now referred to as coroutines (which is a more common term) and the appropriate way to declare them is with the new
@corodecorator. Coroutines simplify concurrent programming when used with Futures and Channels.
- Significant performance improvement of coroutines, now implemented using the greenlet package.
- Channels: can establish channels between arbitrary pairs of chares, and use them to send/receive data inside coroutines. Channels can simplify concurrent programming in some situations.
- Sections: it is now possible to split, slice and combine chare collections in arbitrary ways, to form new collections called sections, containing some subset or combination of elements. Sections are referenced by section proxies, and the usual operations of broadcast and reductions are supported (uses efficient multicast trees that only involve the PEs where the section elements are located).
- Can now create Groups that are constrained to certain PEs, for example:
g = Group(MyChare, onPEs=[0,4,8...]).
- Keyword arguments are now correctly supported when calling remote methods.
charm.wait(awaitables)that waits for the given objects to become ready (works for futures and channels).
charm.iwait(awaitables)that iteratively yields objects as they become ready (works for futures and channels).
- Huge rewrite and revision of examples to use new features (like coroutines and channels), and added documentation and comments. New examples added.
- Same applies to the tutorial and large parts of the documentation.
- Added ARM support (tested on Raspberry Pi 3 B+).
Pool: added an interface to create single tasks. Tasks themselves can spawn any number of other tasks. Python functions decorated with
@corocan be used as tasks that can wait for messages, other tasks to complete, etc.
- Added Quiescence Detection (QD) support. Please see the manual and tutorial for more information.
- Can now use Futures and proxy methods as callbacks, that is: callable objects that can be sent to other chares, and which those chares can call to send values back. For example:
someproxy.somemethodis a valid callback that can be sent (requires Python 3+).
ret=Truekeyword argument now has the same semantics for broadcast calls as for point-to-point calls: the future will receive the return values. For the broadcast case, it will be a list of return values (of all the called elements), sorted by element index.
awaitablekeyword argument to proxy calls, to wait for completion without sending return values to the caller.
- Better method to update globals:
charm.updateGlobals(dict, module_name)is a remote method that can be called to update global values on any PE at any time (typically used as a broadcast call). It has the same rules and semantics as any other proxy call, so it can be waited upon.
- Distributed exception handling: if a future is requested when invoking remote methods (using
ret=True) and an exception happens in the remote method, it is propagated to the caller.
Interactive mode: several QOL improvements. Note that we consider this mode to still be in beta, and encourage feedback. Some highlights:
- Improved launch process for interactive mode. Now also works with charmrun in ssh mode (which can be used to launch an interactive session using multiple hosts).
- Can automatically broadcast import statements to other PEs (on by default).
- Can automatically broadcast and register Chare definitions (on by default).
- Exceptions (whether local or remote) should not crash the interactive session anymore (thanks to distributed exception handling mentioned above).
- Chares now have a method called
migratedthat applications can redefine to get notified when a chare has migrated.
Chare.setMigratable(bool)to indicate whether a chare that is part of an Array can be migrated or not.