Dyalog ’10

Microsoft .NET is a software platform which is based on a component that Microsoft has named the Common Language Runtime (CLR). As the name suggests, the CLR was designed in co-operation with a diverse group of language implementers, and the services that it provides are intended to be "language agnostic". The CLR provides an application virtual machine with memory management, exception handling and other services which significantly simplify the task of implementing new programming languages. It also facilitates communication between modules written in different languages by forcing everyone to use a single memory manager and type system.

The services provided by the CLR make it easy to implement a new programming language, and the ability to inter-operate with solutions written in other languages is very attractive to application developers. However, taking full advantage of the shared type system and related services which not only allows data to be exchanged between programming languages, but also allows the application stack to consist of functions written in different programming languages, forces us to abandon some of the most central dogma of "classic" APL interpreters:

• The notion that APL only has two data types: numbers and characters
• That arguments are always passed "by value"
• User-defined names are global by default, and local variables are visible to all sub-functions

The paper discusses the design of APL# (pronounced APL Sharp), a new dialect of APL designed with object-oriented/language-agnostic platforms in mind, using Microsoft .NET as the initial target platform. Although portability of old APL code to APL# is an important consideration, the fact that complete upwards compatibility with "classic" APL is not achievable allowed us to make an attempt to tidy up a few other aspects of APL. The goal has been to produce a language which is as powerful a Tool of Thought as classic APL and APL2, at the same time as feeling significantly more acceptable to a software engineer.

APL systems provide a definition mechanism so that expressions may be collected into non-primitive or "user-defined" functions and operators: the traditional function or "T-Fn".

In 1996, Dyalog introduced a purer direct-definition style, now referred to as a "dfn", which was designed to fit better with the functional programming paradigm.

APL#, pronounced "APL Sharp", is a new dialect of APL, which is aimed at the Microsoft .NET and similar virtual machine frameworks.

This paper details an attempt, in Dyalog's APL# project, to combine both tradfn and dfn definition styles into a unified whole, which supports both the procedural and functional modes of programming.

A fundamental component of scripting languages (Perl and Tel, for example) is the ability to search text using regular expressions – a means of describing complex patterns of characters within text. Although APL is traditionally used to process numeric data it also has the characteristics of a good scripting language, and current and future APL applications could benefit from the power and flexibility offered by support for regular expressions which is tightly integrated into the language.

This paper discusses the design decisions which led ultimately to the system operators which integrate regular expressions into Dyalog APL. With these, APL users can search text and make modifications, using either a simple expression or a powerful APL function to express the transformation.

We consider the problem of x?y where x and y are real (64-bit floats) or complex (64-bit floats of the real and imaginary parts), with tolerant comparison (nonzero ?CT). We show how to use hashing to solve the problem.

Hashing algorithms for real and complex arguments have been implemented in Dyalog v13.0. APL models and test cases are provided. Benchmarks demonstrate the improvement over the previous implementation.

One of the challenges currently facing software developers is to take advantage of the parallel hardware that is appearing not only in large data centres, but also on every desktop. In addition to the fine-grained parallelism provided by primitive functions which operate on large arrays without loops, APL language interpreters offer a number of "coarse-grained" parallel constructs, which allow multiple invocations of user-defined functions to be expressed. Three examples are each (¨), which is available in virtually all modern systems, rank (?), implemented in SHARP APL and J, and the dot in Dyalog APL (°) which, when placed between an array of objects on the left and an expression on the right, applies the expression to its right to each of the objects on the left.

The paper describes a number of experimental user-defined operators named Peach, Prank, Pouter and Pdot, models of potential extensions to APL interpreters, which allow us to experiment with the performance characteristics of parallel execution using multiple cores in one or more co-operating machines.

Dyalog component files are maintained by the interpreter using indices and other high-level structures within an underlying native file, which is often shared by many users on multi-user operating systems and networks. Each component file update (for example, ?FAPPEND or ?FREPLACE) requires a number of updates to different parts of this underlying file; once started, and until the process is complete, the file may be in an inconsistent state. The series of updates must appear to be atomic to other users and locks are used to ensure that files are not accessed by other users when the file state is inconsistent.

Component files have worked in this way – and generally served APL well – for many years. But many APL users have also experienced damaged component files – ones which are impossible to read or update, which crash the interpreter or, worst of all, present no ill effects but return corrupt values. Almost always the cause of such damage will have been the interruption of a file update (perhaps due to a kill signal to the application making the update or a network failure), leaving the file state inconsistent.

Dyalog APL has addressed this problem in two stages, resulting in component files which are highly robust and resilient to damage. The first stage protected the file from damage caused by interrupting an application during an update and introduced journaling to the process. The second stage dealt with the damage which results from the loss of file caches following a catastrophic event such as a power failure or operating system crash during an update and extended journaling to include cache flushing and checksums.

In addition to the technical details of component file structure, the critical points during an update sequence and the methods used by Dyalog to protect and repair files, this paper will also discuss migration of existing files and compatibility between Dyalog releases and consider the benefits of journaling versus the additional processing involved – so that the best journaling options may be selected for any particular application.

All of the GUI interfaces for Linux are built on X-windows. X-windows is a client-server design. The screen-keyboard-mouse side is the server. Programs running on any machine use the services provided to receive input and draw output.

There is an extension to X-windows servers called xkb which provides additional keyboard support.

This paper describes how Dyalog have used this feature to implement an APL keyboard that overlays the original keyboard. Thanks to the widespread adoption of Unicode for encoding text – including APL symbols – the APL character set can now be entered into almost all applications, whether the underlying keyboard is U.S., Russian – or any other language.

In this presentation we look at the current trends in software development and how the rest of the software world is doing. Some areas where APL is particularly competitive are identified and a list of selling points for APL is presented.

The Dyalog Remote Integrated Development Environment (RIDE) is a new graphical development environment for all versions of Dyalog APL on all platforms – from Windows Mobile to the largest AIX midframes (and, in the future, also the new APL# interpreter). The RIDE allows you to connect to a Dyalog session from almost any web browser on any client platform. The RIDE is a cornerstone of a strategy which is intended to take the portability of applications written in Dyalog APL to the next level.

Morten will review Dyalog APL Version 13.0, which is about to enter Beta Test, and talk about APL#, the RIDE and the rest of Dyalog's Road Map for 2011 and beyond.

Jay is the newest arrival at Dyalog. He secured the job by juggling during the job interview, while talking us through C code which interpreted a "Domain Specific Notation" for juggling known as Vanilla Siteswap (with synchronous + multiplexing extensions). Jay will be presenting his new VS/ APL interpreter.

Dyalog has developed a prototype of a framework which allows your application to look, taste and smell like a relational database. The SQAPL Server allows reporting on live APL data and analytical output from tools like Microsoft Excel or – Access, Crystal Reports – and virtually any programming language – in fact, any client application which can use ODBC, JDBC, ADO or ADO.NET. The interface fully supports SQL, including database updates and inserts, your APL functions as "stored procedures" and creation/modification of tables (DDL statements) – and requires very little SQL expertise to set up.