In accordance with tradition, Morten looks briefly back over his shoulder before turning his gaze to the future, presenting his view of the road that lies before Dyalog and users of Dyalog APL.
John revisits token-by-token tracing (which will be in Dyalog v20.0, promise!), bringing us up-to-date with what it does and what's been added since last year.
Assignment and using the value of a variable are important elements of programming, but, in more complex systems, the name of a variable is sometimes dynamic, and determined separately. Current solutions to these situations suffer from poor readability and performance, so Adám proposes an alternative approach.
Using ⎕SH/⎕CMD to run commands through the operating system's shell is a useful feature, but unfortunately this has some limitations. Peter explores these, and presents a new system function that aims to eliminate them and allow the user to control input output redirections, just like in a real shell.
Richard reveals a few of the interpreter's secrets as he explains what's inside a workspace and how it is managed. This could give you an advantage when tuning your applications for size, performance, or both.
At Dyalog '22, Karta presented an introduction to work that he was undertaking to improve the performance of set-related primitives. He now presents the results of his work and explores what can be expected in upcoming versions of the interpreter.
Following his presentation on Projects and Packages at Dyalog '22, Morten demonstrates a version of the Cider project management system that simultaneously supports two package managers – Tatin for packages implemented in Dyalog, and NuGet for .NET components.
In addition to Dyalog's language-level tools (for example, system functions), there are also many APL-based tools available for general programming, system interaction and interoperation, and application development tasks. Brian explains the efforts being made to make our APL-based tools more readily available, easier to learn, and easier to incorporate into your applications.
D10: Running Dyalog on ARM64 Machinessummary Ron Murray
With Dyalog v19.0, Dyalog can now run on the Apple Macintosh M1 and M2 machines, the Raspberry Pi-4, and Arm 64 processors in the AWS and the Microsoft Azure cloud environments. In this presentation, Ron demonstrates Dyalog running on the Raspberry Pi-4 and in the AWS environment.
Apache Kafka is a distributed event streaming platform that allows you to build real-time data pipelines and streaming apps. It's horizontally scalable, fault-tolerant, and incredibly fast, with latency measured in milliseconds. It's often used as the backbone in microservice architectures in the cloud. Stefan describes his initial experiments on what a Dyalog interface to Apache Kafka might look like.
The key operator (⌸), introduced in Dyalog version 14.0, is arguably the most successful recent addition to the language, enabling data scientists and finance processors to group, summarise, and count values in various ways. However, sometimes key disappoints because it doesn't know the intended order of results, is missing placeholders for absent keys, or includes unwanted data. Adám proposes a simple extension that addresses these issues.
Back by (allegedly) popular demand, Andy will once again delve into the lesser-known aspects of Dyalog and our backend systems – he's got several versions to talk about!
The Co-dfns project is an initiative to expand the reach and capabilities of APL through compiler design, language research, and the use of APL to tackle problems that have been previously under-represented in the modern APL community. Aaron provides an update on these efforts, including the status and current capabilities of the compiler, a roadmap for the future, and practical details concerning how Co-dfns can be applied today.
Aaron's Co-dfns presentations at previous Dyalog user meetings:
With Data Analytics and Machine Learning on the rise, it is now critical to have statistical libraries available. Josh takes a look at some of the open-source statistical libraries available to the APL developer.
The 15th annual International APL Problem Solving Competition took place earlier this year. Brian summarises this year's contest and introduces the grand prize winner, culminating in the award ceremony.
Vega-lite is a high-level data visualisation grammar - a declarative language for describing visual presentations in JSON (JavaScript Object Notation). Charts are rendered using a JavaScript library and, thanks to the HTMLRenderer, we can provide data directly from APL and produce graphics ready for publication on the web. Rich demonstrates a simple interface for producing graphs from APL data using Vega-lite.
Dyalog Ltd is growing and times are changing. Stine presents some of the work going on in the background to help us evolve in the years to come.
Licensing can be a very complex subject, and it can be difficult to determine what your exact licence requirements are. We've been looking at simplifying our current structure to make the process more intuitive. Jada presents the changes to licensing that will take effect in January 2024.
Stig demonstrates a small “WorkerBees” project that enables you to run APL tasks in parallel on separate threads using the concept of “hosted APL” and .NET async/await and tasks. The aim is to create reusable “APL worker bees” that can be dedicated to certain tasks, where “parallel each” is short lived thread processes.
BIG is a company in the jewellery industry that integrates with dozens of systems to import data from 1,000 retailers. The data is used to provide inventory analysis and merchandising consulting for the retailers, to provide sell-through data to manufacturers, and to integrate to several industry service companies. The end-user tools, and the integration applet, are written in C#, ASP.NET, JavaScript, MSSQL, and KendoUI.
At Dyalog '22, Mark introduced us to BIG's new Dyalog-based production environment. Dyalog's tools and features, including SQAPL, Jarvis, HttpCommand, .NET integration, ⎕JSON, and ⎕CSV, have enabled BIG to address specific aspects of its processes, and provide the ability to respond to customer needs faster and more flexibly than ever before. Mark updates us on how BIG's use of Dyalog and its tools has grown in the last year.
U03: The Evolution of APLTree and APL-cationsummary | slides (ZIP) | video (25 mins) Kai Jaeger, independent (Germany)
At Dyalog '18, Kai presented the evolution of the APLTree and the APL-cation projects. In the five years since then, all APLTree projects have been transformed into Tatin packages. Kai discusses the benefits and implications of this change, and updates us on the latest changes and additions to these open-source projects.
Ray muses on about a decade of trying to play J.S Bach's "Toccata and Fugue in D minor, BMV565" using APL. His musings include sound "WAV" file format and their creation, what makes up a chord, (adding of 3 sine waves), what harmonics are and how they can be combined to convert a simple tone into the sound of organ pipes, and adding reverberation to a tune to give the effect of a large concert hall. He culminates in his rendition of BNV565 in full stereo, with an animation showing each note as it is played.
DPC recently migrated an application from APL+Win to Dyalog. Finn describes the process, particularly the conversion of a COM Server to a Jarvis-based web service and the deployment of the application as a Docker container.
Claus has put a complex financial application in the cloud and made it available to clients anywhere in the world. He explains how he – much to his surprise – ended up enjoying his journey from saved workspaces to the automated building and deployment of docker containers to Amazon Web Services.
Quantum computing is a promising technology emerging from harnessing the laws governing the world of atoms. Writing quantum programs equates to specifying circuits, which is cumbersome and does not scale intellectually. We have found at NCSA that APL naturally expresses quantum concepts, leading to higher level programming constructs. Marcos demonstrates these connections and how they might help to revolutionise this form of computing.
The capacity of APL enables domain experts to solve difficult problems makes it the perfect candidate to support scientific and engineering work. In the field of materials science and metallurgy, there are a large number of analysis and modelling problems that could be successfully tackled with the help of APL. Moreover, APL has traditionally been not only a programming language with useful applications, but also a powerful learning tool. Jesús shows us how APL is being used in the field of metallurgy, in an academic environment, both for education and research.
As part of a Dyalog internship over the summer of 2023, Asher has been developing training courses for teaching linear algebra using APL to university students with the aim of spreading the APL message. Asher describes his Dyalog experience and how he thinks the first training course went.
YAML is a human-readable data-serialisation language. Brandon has written a YAML parser with an architecture that resembles that of Co-dfns (a high performance compiler for a parallel extension of APL's dfns). He discusses the parser implementation, the particular design decisions made, and how a linear data-flow model enables him to write APL that cuts across all levels of detail simultaneously.
Suppose some students are confronted with a relatively simple programming problem in a computational materials course, like writing a grain growth model. These students are not software developers, but future materials scientists. What tools do they know to help them with this task? Could they benefit from using array programming techniques? What about APL? Jesús answers these questions by analysing a real, practical, case.
Alexander is the winning professional entrant of the 15th annual APL Problem Solving Competition. He describes what led him to Dyalog APL and the competition, and explores some of the approaches leading to the solutions behind his successful entry.
Andrea is the grand prize winner of the 15th annual APL Problem Solving Competition. Following the award ceremony, he describes his experiences of APL/Dyalog and tells us how he managed to master enough of it to win the competition.
Learning APL can be both exhilarating and excruciating. Most pedagogical materials aim at the early intermediate level or lower; other materials are aimed at experts. Brandon offers the perspective of a firmly middle-intermediate APLer trying to make the most of the glittering promises of "APL as a tool of thought" while the pains and joys are salient.
Array Notation is a convenient method for introducing APL array and namespace literals in APL code. Proposals for its syntax and implementation were first made by Phil Last in 2015, and have been discussed and refined for implementation in Dyalog over the last several years. As part of a Dyalog internship over the summer of 2023, Kamila has implemented support for Array Notation in the interpreter and now presents performance benchmarks for the new syntax.
Presentations on Array Notation at previous Dyalog user meetings:
SA1: Project Management using Link, Cider, and Tatin (Part 1 of 2)summary | slides (PPTX) | slides (PDF) Morten Kromberg, Kai Jaeger, and Gilgamesh Athoraya
Recent versions of Dyalog include Link, which makes it easy to use a folder containing text files as the source of your APL code, rather than a binary workspace. That might be all you need for really simple applications, but it doesn't take long before you want to have dependencies on code from other projects – or Tatin packages – to complete your application. You might also want to have tests and development tools that are in your source folder(s) but not delivered to end users. These additional features are the domain of a project management system like Cider.
This workshop provides an introduction to using Cider (which is included with Dyalog version 19.0) to manage Dyalog APL projects.
If you are interested in making your work available to other users in the form of a package, or simply want to understand how packages are created, you can continue with second part of the workshop on publishing packages using Tatin (SP1).
Keywords: project management, Tatin, Cider, Link
SA2: Tacit Techniquessummary | slides (PPTX) | slides (PDF) Adám Brudzewsky and Rich Park (asst. Peter Mikkelsen)
Are you intrigued or confused by the growing prevalence of tacit APL code (trains and compositional operators)? If so, this workshop is for you! We'll help you to spot relevant patterns, guide you in using appropriate tacit constructs to your advantage, and coach you how to decipher others' code if you find it hard to read at a glance. In addition, we'll show you functionality and performance enhancements that can easily be achieved with tacit techniques, expanding your vision of what's possible with APL.
Keywords: function composition, trains, tacit, operators
SA3: Applied Performance Challengesummary | all materials (GitHub) Karta Kooner and Aaron Hsu
This workshop is designed to encourage a holistic understanding of how to think about and improve the performance of APL code. You will be given the opportunity to collaboratively apply a variety of techniques at all scales of performance optimisation to a non-trivial APL application.
In this hands-on workshop, you will be able to traverse through all the phases of application optimisation to understand how each phase impacts the others. We intend to cover both high level and low level optimisation, as well as high and low level performance analysis.
SP1: Project Management using Link, Cider, and Tatin (Part 2 of 2)summary | slides (HTML) Gilgamesh Athoraya, Kai Jaeger, and Morten Kromberg
Following on from the workshop on APL project management (SA1), we will look at the process of creating a package from a project, publishing it to a Tatin server, and releasing new versions.
Dyalog's graphical integrated development environments (the cross-platform RIDE and Microsoft Windows-only IDE) have many useful features that can enhance your ability to develop and debug code, not all of which are immediately obvious. We'll take you on a hands-on tour in which you'll be able to use existing functionality and add your own, making you a more effective APLer.
A web service is a software system designed to enable communication and interoperability between different applications or systems over the internet. It provides a standardised way for different software applications to exchange data and perform various functions. Web services are commonly used for a wide range of purposes, such as integrating different systems, sharing data between applications, automating business processes, and building distributed applications. Web services provide a scalable and platform-independent way of enabling communication and data exchange between disparate software systems. These functionalities can be accessed and utilised by other software applications, irrespective of the programming languages or platforms they are built on.
In this hands-on workshop, you'll learn how to:
consume web services using HttpCommand. HttpCommand is a utility used to issue HTTP requests and receive responses. It can interact with practically any web service. We will:
cover the most commonly used features of HttpCommand
learn how to develop and debug web service requests
access a variety of web services including web services that require authentication
provide web services using the Jarvis web service framework. Jarvis makes it easy to make many Dyalog applications available over the internet. We will:
develop a simple web service in 5 minutes
cover the essential features of Jarvis
examine and experiment with a sample web service
demonstrate how to access our web service from a browser and using HttpCommand
Writing your own test framework in APL only takes a few minutes, which is one of the reasons why there are so few shared frameworks – most users prefer to build something that is tuned to their own taste and specific internal requirements.
Whether you are starting a new APL project or it is time to revisit the way you maintain an existing system, it is worth taking a look at best practices in the software engineering community. Many modern practices also make good sense for APL development, and can be easily adopted. This workshop will begin with the introduction of a minimalist testing framework, present Dyalog's own "DTest" framework, and end with a demonstration of how you can establish an "industry standard" continuous integration pipeline using GitHub.
TP2: Leading Axis Theory and Practicesummary | slides (PPTX) | slides (PDF) Rich Park and Adám Brudzewsky (asst. Josh David)
This hands-on workshop will show you how to design your functions and structure your data so that you can wield the keys to the array kingdom – the rank operator and dyadic transpose. Rich and Adám will demonstrate how these, together with various other leading axis primitives, can simplify your code and give you superior performance. You'll be presented with various problems to solve, as well as participate in a discussion on how to maximise benefits from leading axis theory.
Keywords: rank operator, transpose, leading axis theory, performance
TP3: APL Functions to Import, Export and Process Data in Filessummary | slides (PDF) | all materials (GitHub) Richard Smith and Peter Mikkelsen
The interpreter provides several built-in functions to import, export, and process data in files, and to manage the host file system. These are invaluable tools in their own right, and can be the building blocks for powerful APL applications that are largely independent of the host system on which they run. This workshop examines these functions in depth, starting with a simple introduction and introducing their Dyalog v19.0 enhancements.
