yibie/excali-mode

#+title: excali-mode — Excalidraw inside Emacs

#+html: <p align="center"><img src="hero/hero.gif" alt="Drawing, binding arrows and growing a flowchart in excali-mode" width="800"></p>

excali-mode opens, edits and saves [[https://excalidraw.com][Excalidraw]] drawings in Emacs.  The
=.excalidraw= files are the same ones the web app reads and writes, and
the drawing looks the same: rough.js strokes and hachure fills, the
hand-drawn fonts, and arrows that stay attached to shapes.  Elisp holds
the scene and every editing command; a small C module draws it with
Cairo and Pango straight into the pixel buffer of Emacs 32's new
=canvas= image type.

It is an unofficial port, not affiliated with the Excalidraw project,
and it needs Emacs 32 (master) — the first Emacs with the canvas API.

* Features

- Every Excalidraw tool: rectangle, diamond, ellipse, arrow, line, pen,
  text, image, frame, sticky note, eraser, laser pointer, eye dropper,
  bucket fill, autoshape and lasso selection.
- Arrows that bind to shapes and follow them, including elbow arrows
  routed around obstacles; =Cmd+arrow= grows a flowchart node by node.
- Selection, groups, resize and rotate handles, align and distribute,
  flip, lock, snapping and grid, z-order, copy and paste of elements and
  styles, undo and redo.
- Labels in shapes and on arrows, text wrapping and fonts as upstream
  lays them out (CJK included), dark mode.
- Images (PNG, SVG, JPEG, GIF, WebP), frames with clipping, the element
  library (=.excalidrawlib=) with the official collection of
  libraries.excalidraw.com one command away, and PNG/SVG export with the
  scene embedded so Excalidraw can open the export again.
- Diagrams from text: the =.edsl= language of [[https://github.com/tyrchen/excalidraw-dsl][excalidraw-dsl]], laid out
  automatically and drawn with bound arrows and labels you can go on
  editing.
- Smooth on large scenes: only damaged areas repaint, panning reuses
  pixels, zooming shows an instant preview, and on macOS a CoreAnimation
  layer bypasses Emacs's image refresh.

* Installation

excali-mode is not on MELPA yet.  Build it from a clone:

#+begin_src shell
git clone https://github.com/yibie/excali-mode.git
cd excali-mode
make            # the C module and the byte-compiled Lisp; see Platforms
make fonts      # optional but recommended: Excalidraw's fonts (network)
#+end_src

Run =make= again after pulling: excali loaded from uncompiled =.el= files
runs several times slower, which large scenes feel.  Emacs picks the
=.elc= files up by itself, and native-compiles them in the background
where it can.

#+begin_src emacs-lisp
(use-package excali
  :load-path "/path/to/excali-mode"
  :commands (excali-open excali-new))
#+end_src

=M-x excali-open= opens a =.excalidraw= file (or a PNG/SVG exported with
an embedded scene); =M-x excali-new= starts an empty drawing.

* Build and test

#+begin_src shell
make            # builds excali-module (.so/.dylib/.dll per Emacs; needs cairo, pangocairo, zlib)
make compile    # byte-compiles the Lisp (part of make)
make info       # shows the detected platform, Emacs, header and module name
make test       # batch ERT: round-trip, text, rendering, tiles, resizing
make try        # opens the sample in a clean GUI Emacs
make fonts      # downloads Excalidraw's fonts into fonts/ (see fonts/README)
make bench      # benchmarks backends in a clean GUI Emacs, writes bench.txt
#+end_src

** Platforms

All platforms need an Emacs 32 (master) built with module support —
its =emacs-module.h= must have =canvas_data= — plus GNU make,
pkg-config, a C11 compiler, cairo, pangocairo and zlib.  librsvg,
gdk-pixbuf and libwebp (SVG, GIF/JPEG and WebP images) and fontconfig
(the fonts in =fonts/= on Linux) are used when pkg-config finds them;
=EXCALI_WITH_RSVG=no= etc. leave one out.  Set =EMACS=/path/to/emacs= to
build against a particular Emacs; the header is looked up next to it
(=../include/= when installed, or =src/= in a build tree), else set
=EMACS_MODULE_INCLUDE=.

- macOS (Homebrew): =brew install pkg-config cairo pango librsvg
  gdk-pixbuf webp=.  Adds the CoreAnimation =layer= backend and
  registers fonts with CoreText.
- Linux, Debian/Ubuntu: =apt install build-essential pkg-config
  libcairo2-dev libpango1.0-dev zlib1g-dev librsvg2-dev
  libgdk-pixbuf-2.0-dev libwebp-dev libfontconfig-dev=; Fedora: =dnf
  install gcc make pkgconf cairo-devel pango-devel zlib-devel
  librsvg2-devel gdk-pixbuf2-devel libwebp-devel fontconfig-devel=.
  The BSDs build the same way with =gmake=.
- Windows, MSYS2 UCRT64 shell: =pacman -S make
  mingw-w64-ucrt-x86_64-{gcc,pkgconf,cairo,pango,zlib,librsvg,gdk-pixbuf2,libwebp}=,
  with an Emacs 32 built in the same environment.  The module links
  against the UCRT64 DLLs, so =/ucrt64/bin= must be on Emacs' =PATH=.
  Fonts in =fonts/= are registered with GDI (=AddFontResourceEx=).

Without CoreAnimation the =auto= backend is =tiles=.  What has been
checked: the full build and =make test= on macOS; the non-macOS
configuration (=make PLATFORM=unix=, no CoreAnimation or CoreText) built
with GCC and passing =make test= on macOS; every source compiling
cleanly for x86_64 glibc, aarch64 musl and x86_64 mingw-w64 (=zig cc=
with those libcs' headers).  Not yet run on Linux or Windows: linking
there, the GUI, and on Windows whether Pango sees GDI-registered fonts.

* Keys

=excali-open= also opens PNG and SVG files exported with an embedded scene
(by excali or Excalidraw); such scenes are saved to a new =.excalidraw= file.

Plain keys follow Excalidraw; its Mod shortcuts use super (Cmd on
macOS), and the usual Emacs keys work too.  =?= lists everything.

| Key | Action |
|-----+--------|
| =v= =1=, =h=, =r= =2=, =d= =3=, =o= =4=, =a= =5=, =l= =6=, =p= =x= =7=, =t= =8= | select, hand, rectangle, diamond, ellipse, arrow, line, pen, text |
| =a= again | cycle sharp / round / elbow arrows |
| =f=, =n=, =e= =0= | frame, sticky note, eraser (=e= again: previous tool) |
| =k= | laser pointer: a fading red trail, never saved |
| =b= | bucket fill: click inside a closed area (=b= again: next color, =M-click=: pick the color) |
| =X= | autoshape: a freehand stroke becomes a rectangle, ellipse, diamond or line |
| =i= =G=, =S= | eye dropper: next click picks a background, stroke (=M-click=: the other) |
| =C-M-drag= =M-s-drag= | lasso select; =M-x excali-toggle-lasso= makes it the =v= tool |
| =q= | lock the tool (keep drawing) |
| drag | create; shift = square / 15° lines, meta = from center |
| short drag with =a=/=l=, then clicks | multi-point line; click the last point, =RET= or =ESC= to finish |
| click, =S-click=, box drag | select, toggle, box select (contain) |
| drag inside the selection box | move |
| corner handles, border, round handle | resize (shift keeps ratio, meta from center), rotate (shift snaps 15°) |
| double-click | enter group, edit text, add or edit a shape's label, add text |
| =s=, =g=, =F= | style panel (as upstream's; each property opens a panel of its values, colors a picker: =q=..=b= colors, =1=..=5= shades, =#= hex), background, font |
| =H=, =V= | flip horizontally / vertically |
| =TAB= | cycle rectangle / diamond / ellipse |
| arrows, =S-=arrows | nudge 1 / 5 |
| =s-z= =s-Z= (=C-/= =C-?=) | undo, redo |
| =s-c= =s-x= =s-v= (=M-w= =C-w= =C-y=) | copy, cut, paste (Excalidraw clipboard) |
| =s-d=, =s-a=, =s-g=, =s-G= | duplicate, select all, group, ungroup |
| =s-]= =s-[=, =M-s-]= =M-s-[= | forward / backward, front / back |
| =S-s-=arrows, =M-h= =M-v= | align, distribute |
| =s-=arrows (=C-=arrows) | flowchart: add a linked node; repeat for siblings, any other key keeps them |
| =M-=arrows | flowchart: go to the linked node that way; repeat to cycle |
| =s-L= | lock / unlock elements |
| =s-k=, click a link icon | set a link (URL or =?element=ID=), follow it |
| =s-'=, =M-s= | grid, object snapping (super at the press suppresses/inverts) |
| =M-D= | toggle the dark theme |
| =C-c l a=, =C-c l i=, =C-c l b=, =C-c l d=, =C-c l o= | library: add selection, insert items, browse, delete items, official collection |
| =M-s-c= =M-s-v=, =s-<= =s->= | copy / paste styles, font size |
| wheel (=S-=wheel sideways), =C-=wheel / pinch, middle or right drag | pan (the way Emacs scrolls a buffer; =mouse-wheel-flip-direction= swaps sideways), zoom, pan |
| =s-== =s--= =s-0= (=C-x C-= - 0=), =!= =@= =#= | zoom, zoom to fit all / selection |
| =PgUp= =PgDn= (with shift: horizontal) | page |
| =9= | insert an image file at the view centre |
| =s-s=, =C-x C-s= | save |
| =s-E=, =C-c C-e= | export PNG or SVG (by extension; the selection if any, =C-u= for all) |
| =C-c C-b=, =C-c C-p= | cycle backend, toggle 1x/2x (debug) |

* Drawing from text

=excali-dsl-mode= edits =.edsl= files, the language of [[https://github.com/tyrchen/excalidraw-dsl][excalidraw-dsl]];
=C-c C-c= draws the buffer in an excali buffer beside it (again: the
same buffer, same view), =C-c C-e= writes an =.excalidraw= file.  The
layout is layered, like dagre: nodes by rank along the edges, each
container laid out on its own and placed as one block, and straight
edges bent around the shapes in their way.  Everything drawn is
ordinary Excalidraw: arrows are bound and follow their shapes, labels
are bound text.

#+begin_src text
---
direction: LR            # TB (default), BT, LR, RL
nodeSpacing: 60
rankSpacing: 90
---
componentType service {
  shape: rectangle;
  style { fill: "#e3f2fd"; strokeColor: "#1976d2"; }
}

client[Web Client]
container "Backend" as backend {
  api[API] { type: service }
  db[Database] { shape: ellipse; backgroundColor: "#fce4ec" }
  api -> db: query
}
client -> api: HTTPS
api -> cache -> db       # a chain; `cache' is made on first use
db <-> replica @orthogonal
#+end_src

- Nodes: =id=, =id[Label]= or =id "Label"=, then =@type= or a
  ={ key: value; ... }= block: =shape= (rectangle, ellipse, diamond,
  cylinder, text), =backgroundColor= or =fill=, =strokeColor=,
  =strokeWidth=, =strokeStyle=, =fillStyle=, =roughness=, =roundness=,
  =opacity=, =font=, =fontSize=, =textColor=, =width=, =height=, =type=.
- Edges: =->=, =<->=, =--= or =---= (no heads), =~>= (curved); labels
  as =a -> b: text=, =a -> b "text"= or =a -> b {text}=; a style block
  with =strokeColor= or =color=, =strokeWidth= or =width=,
  =strokeStyle=, =startArrowhead=, =endArrowhead= and =routing=
  (straight, orthogonal, curved); or =@orthogonal=, =@curved=.
- Containers, =container "Label" as id { ... }= or =container id "Label"
  {=, and groups, =group=, =flow=, =service=, =layer=, =component=,
  =subsystem=, =zone= or =cluster= followed by a label, nest.  A
  cluster's own =key: value= lines or =style: { ... }= style it, and
  =layout: horizontal=, =vertical= or =grid(N)= arranges members without
  edges.  =backend.api= refers to =api= in =backend=.
- =componentType=, =connection= and =connections= blocks, front matter
  =component_types= and =templates=, and =template= with =layers= plus
  =diagram "Title" { template: name }=.
- Front matter: =direction= (or =rankdir=, =layout_options=),
  =nodeSpacing= or =nodesep=, =rankSpacing= or =ranksep=, =theme= (dark),
  =font=, =sketchiness=, =stroke_width=, =background_color=, =routing=.

From Lisp, =(excali-dsl-elements STRING)= returns the elements and
=(excali-dsl-scene STRING)= a whole document.  In an excali buffer
=M-x excali-dsl-yank= adds the diagram on the kill ring at the view
center, =M-x excali-dsl-insert-file= one from a file.  Errors name
their line.  =docs/dsl.md= is a compact reference of the language.

Not supported: the force and ELK layouts (the layered one is used),
upstream's ML layout, and template layouts other than per-layer
=horizontal=, =vertical= and =grid(N)=.

* Presentation backends

The module renders into its own offscreen framebuffer; =excali-backend=
chooses how that reaches the screen:

- =canvas= — copy every frame into one Canvas image.
- =tiles= — split the window into Canvas tiles; the module diffs each
  tile against its last contents and only changed tiles get
  =canvas-refresh=.  Drags also repaint only the damaged rectangle.
- =layer= (macOS) — a CoreAnimation layer over the Emacs view shows the
  framebuffer via IOSurface, bypassing =canvas-refresh=.  Mouse events
  still go to Emacs.
- =auto= (default) — =layer= on graphical macOS frames when the module
  was built with it, =tiles= everywhere else.  An explicit choice wins.

Independently of the backend, rendering only repaints what changed:
elements outside the view are culled, drags repaint the damaged
rectangle, and panning by whole device pixels shifts the framebuffer in
place and paints only the newly exposed strips (sub-pixel trackpad deltas
accumulate first).  Zooming (C-wheel, pinch, === / =-=) shows a preview
first: the module scales the pixels of the last full render about the
anchor with bilinear filtering (about 0.5 ms at 1x, 2 ms at 2x), and a
crisp full render follows once no zoom step came for
=excali-zoom-preview-delay= (0.1 s; nil renders every step fully).  Steps
more than =excali-zoom-preview-limit= (4x) away from that render render
fully.

A scene shown in several windows gets a view per window, each with its
own zoom, scroll, framebuffer and surfaces; a change made in one window
redraws the others after the command.

=M-x excali-bench-backends= in an excali buffer compares them; =make bench=
does the same in a fresh GUI Emacs and writes =bench.txt=.

* Text and fonts

Text follows Excalidraw's layout exactly wherever it does not depend on
glyph widths: per-family line heights and baselines (=FONT_METADATA=,
=getVerticalOffset=), heights of lines × size × line height, the
tokenizer-based =wrapText= (CJK, emoji, hyphen and whitespace rules),
fixed-width (=autoResize= false) text, and labels in rectangles,
ellipses, diamonds, sticky notes and on arrows (padding, maximum sizes,
alignment, container growth, =labelPosition=).  Pango only measures and
draws single lines, so widths match the web app when the same fonts are
installed.  =make fonts= downloads them into =fonts/=, which is
registered with the font backend (CoreText on macOS with Homebrew
Pango, fontconfig elsewhere) when excali loads; without them text falls
back to system fonts, see =excali-font-families= and
=M-x excali-font-report=.

* Element library

The personal library lives in =excali-library-file= in Excalidraw's
=.excalidrawlib= format, so it moves freely between excali and
excalidraw.com.  =C-c l a= adds the selection, =C-c l i= inserts items,
=M-x excali-library-import= / =excali-library-export= merge and write
library files.

=C-c l b= browses it as thumbnails: =RET= or a click inserts the item
into the scene, =d= deletes it (after asking), =g= redraws, =o= opens
the official collection.  =C-c l d= deletes items by name.

=C-c l o= (=excali-library-browse-official=) lists the official
collection of [[https://libraries.excalidraw.com][libraries.excalidraw.com]]: name, item count, downloads,
last update, authors and description.  =RET= previews a library (the
site's picture and every item), =a= adds all of it, =/= filters by a
regexp over names, descriptions, authors and item names, =g= fetches
the index again, =o= shows the library on the site; =S= sorts by the
column at point; a =✓= marks libraries wholly in the personal library.
The preview takes the list's place: =RET= inserts the item at point,
=+= adds it to the personal library, =a= adds them all, =q= goes back to
the list where you left it.  Items the library holds say =✓ in library=;
added ones say =✓ added= and light up.  Items keep their names; unnamed
ones take the library's.  Adding a library twice adds nothing twice.

Thumbnails are cached as PNG files in
=excali-library-thumbnail-directory=; those not cached yet show as grey
squares and are drawn in the background, those in view first, so large
libraries open at once.  The index is fetched from
=excali-library-official-url= (the files the site's "Add to Excalidraw"
button hands to excalidraw.com) and cached in
=excali-library-official-cache=.

* Images, frames and export

Images show =files[fileId].dataURL=; each file is decoded once per
session in the module and freed when no excali buffer uses it.  PNG is
decoded by Cairo; SVG needs librsvg, WebP libwebp, and JPEG, GIF (first
frame), BMP, ICO etc. gdk-pixbuf.  These are optional: the Makefile uses
whichever pkg-config finds (disable with =EXCALI_WITH_RSVG=no=,
=EXCALI_WITH_PIXBUF=no=, =EXCALI_WITH_WEBP=no=), and undecodable images show
upstream's error placeholder.  Flips (=scale=), =crop=, rounded corners
and opacity follow upstream.  =excali-insert-image= ids files by SHA-1 and
scales raster images larger than =excali-image-max-size= (1440) down, as PNG.

Frames draw upstream's outline (2px, radius 8, constant on screen) and
their name above (14px, cut with an ellipsis), clip their children and
multiply their opacity into them.

=excali-export-png= / =excali-export-svg= follow upstream's export:
padding =excali-export-padding= (10), scale =excali-export-scale= (1–3),
=excali-export-background=, frame names as Helvetica 14px text, a single
selected frame exported alone, and =excali-export-embed-scene= (on by
default) embedding the scene as upstream does (PNG =tEXt= chunk
=application/vnd.excalidraw+json= with a zlib-compressed payload; SVG
=<metadata>= payload version 2).  The SVG is Cairo's (text as glyph
paths, images embedded as PNG) with upstream's root size and metadata.
No dark-mode export yet.

* Layout

- =src/excali-render.c= — roughjs-style strokes, hachure fills, Pango text.
- =src/excali-text.c= — font registration, Pango line measuring and drawing.
- =src/excali-module.c= — module glue, framebuffer, tile diffing.
- =src/excali-preview.c= — zoom previews scaled from rendered pixels.
- =src/excali-layer.m= — CoreAnimation overlay backend (macOS).
- =src/excali-image.c= — image decoding, the image cache, image and
  placeholder drawing.
- =src/excali-frame.c= — frame outlines, names and child clipping.
- =src/excali-export.c= — PNG/SVG export surfaces, tEXt chunks, zlib.
- =src/excali-fill.c= — bucket-fill region search: raster flood fill,
  gap bridging, outline tracing with keyholed holes.
- =excali-core.el= — module loading, shared state, document model, file
  reading and saving (JSON written like upstream's =JSON.stringify(…, null, 2)=).
- =excali-restore.el= — restore/migration of loaded scenes (port of
  upstream =restore.ts=); unknown element types and fields are kept.
- =excali-index.el= — fractional z-order indices (=fractional-indexing=,
  =syncInvalidIndices=, =syncMovedIndices=).
- =excali-text.el= — fonts, text measurement, wrapping, bound text and
  arrow labels.
- =excali-view.el= — framebuffer, presentation backends, damage, scroll
  reuse, pan/zoom.
- =excali-select.el= — selection model, groups, box selection, overlays.
- =excali-handles.el= — selection UI geometry, transform handles.
- =excali-transform.el= — resize / rotate math.
- =excali-hit.el= — hit testing.
- =excali-create.el= — drawing tools, click-click lines, tool lock.
- =excali-actions.el= — flip, align, distribute, lock, styles, zoom to fit.
- =excali-binding.el= — arrow binding, and =excali--follow= (arrows and
  labels following moved shapes).
- =excali-linear.el= — point editor for lines and arrows.
- =excali-snap.el= — grid mode and object snapping.
- =excali-frame.el= — frame membership and behavior.
- =excali-erase.el= — eraser and element links.
- =excali-tools.el= — laser, eye dropper, autoshape, lasso.
- =excali-bucket.el= — bucket fill.
- =excali-library.el= — =.excalidrawlib= library: add, insert, import,
  export, browse, delete; the official collection.
- =excali-edit.el= — drags, mouse commands, text, grouping, z-order.
- =excali-cursor.el= — pointer shapes (upstream's CSS cursors; on macOS the
  module shows them over the canvas, elsewhere the nearest Emacs =pointer=).
- =excali-history.el= — undo/redo with shared per-element snapshots.
- =excali-clipboard.el= — copy/cut/paste/duplicate.
- =excali-image.el= — image render data, the files map and the module's
  image cache, =excali-insert-image=.
- =excali-frame-render.el= — frame titles, name label bounds, frame
  render data.
- =excali-export.el= — PNG/SVG export and import of embedded scenes.
- =excali-dsl.el= — the =.edsl= diagram language: parser, model, drawing,
  =excali-dsl-mode=.
- =excali-dsl-layout.el= — layered graph layout (cycle breaking,
  layering, crossing reduction, least-squares placement).
- =excali-bench.el= — backend benchmarks.
- =excali.el= — keymap, major mode and entry points.

* Known gaps

Not ported: collaboration, AI / magic frames, embeddables, Mermaid, the
app shell, and interactive image cropping (a stored =crop= renders).
Autoshape recognition is a heuristic rather than upstream's
=convertToShape=.  Text is typed in the minibuffer with a live preview
rather than on the canvas.  Rough strokes resemble but do not match the
web renderer pixel for pixel.  Without =make fonts=, text uses system
fallback fonts, so widths and wrapping differ from the web app.  Native
pointer shapes need macOS; elsewhere the nearest Emacs =pointer= shape
stands in.  Linux and Windows builds are cross-compiled but not run.

* Changes

See =CHANGELOG.org=.

* Acknowledgements

excali-mode was indirectly inspired by [[https://github.com/0WD0/video.el][video.el]],
which uses Emacs's Canvas API to display images and video inside the
editor.

* License

GPL-3.0-or-later; see =LICENSE=.  excali-mode ports code from Excalidraw,
roughjs and perfect-freehand (MIT) and fractional-indexing (CC0); their
notices are in =THIRD-PARTY-NOTICES=.  The fonts =make fonts= downloads
keep their own licenses, listed in =fonts/README=.