Workshop

The workshop started with two short talks about the early visions of home IoT and thing-centered design approaches for the future smart home. We focused on the “Awareness of Things” introduced by dr. Yu-Ting Cheng, a vision of object-centered and object-mediated intelligent experiences in the Everyday. What does this mean? Commonly, smart objects and connected devices are connected to the Internet of Things (IoT), to cloud services provided by device manufacturers, so that we can control devices in the home with our smart phones. IoT data sharing is used to backup information in the cloud, but mostly to analyze and learn about our Everyday. In this sense, connected devices share a lot of data with external parties, but do very little to connect with other devices in the context. Long story short, we are missing out because our smart objects in the home don’t talk to each other. Design did that, design can change that.

Awareness of Things turns this premise around and encourages designers to think from the object perspective, framing smart objects in a context as a collaborative ecology with shared goals and values. Objects could help each other sense changes in the environment, or they could work together on tasks where some objects provide information, and others carry out physical action. Awareness means that objects deliberately share information with other objects in the same context, often subdivided into small scopes such as floors or rooms. While this design approach can inspire completely new use-cases, we can also leverage collective objects to make existing user interactions and user experiences more seamless and supportive.

The participants in the workshop worked in teams of four students from architecture, communication design, fashion design and product design. Every team had to choose a context of interest for which they would design several different scenarios of objects that collaborate in novel ways. The teams had little more than a week to design and prototype not only several objects, but also props and a stage for their context, so that visitors of the final exhibition could experience the object collaboration with working interfaces and interactions. All objects that the teams would design had to communicate live with each other and exchange data to realize the desired user experience.

Technology

For this purpose, we worked with a shared middleware, called OOCSI, which allows for easy prototyping across platforms and data sharing through simple messages. OOCSI has been developed in the Industrial Design department of Eindhoven University of Technology specifically for design prototyping and as an open-source solution. In addition, we prepared an educational framework of ready-made sensing or actuating “things” that could be used to bootstrap the system design process. These things were implemented as interactive HTML pages, using P5.js and OOCSI, so they could be opened on students’ smart phones, tablets, and laptops, and provide building blocks for “sketching” data-sharing experiences without even touching any code. Every single thing had a catchy name. For instance, “Lampo”, showing a large rectangle that could be toggled by tapping on the screen between bright and dark grey, or “Touchee” that would allow for swipes and touch input. By opening Lampo on a smart phone and creating a simple holding enclosure for the phone, we can create a simple table lamp. If we would open the same page on a different smart phone, the two DIY lamps would be linked: a tap on one of them would toggle both. Other things would activate the phone camera or microphone and report a color or sound volume. Yet others could work with the internal acceleration sensors of a smart phone or visualize data. In total, we built 20 different “things” for this workshop, and they provided a new data-enabled design space for the workshop. Next to these building blocks, the students designed physical prototypes, which were built on ESP8266 or ESP32 microcontrollers and connected to the local WIFI and OOCSI network. All physical “things” communicated seamlessly with the ready-made things we prepared, allowing the teams to work with tilt sensors, servo motors, and LED lights to realize the different contextual experiences.

Design process

With this emerging technology and new object-centric way to prototype, the students chose diverse contexts for their explorations: from the public library to the home, from different office experiences to a public toilet, they thought about what things and connected objects mean for the design of new experiences in the Everyday. The teams first explored the real-life contexts and made a photo documentary. They sketched new experiences, changing routines, sometimes came up with absurd or funny situations for mundane scenarios. Several alternating presentation and prototyping sessions helped them narrow their concepts and move fast towards experiential prototypes. This involved acting out users and objects, creating short movies and making storybooks of alternative scenarios. The teaching team provided feedback and prototyping support, both in the early phases when students needed ideas on how to realize particular objects or interactions, and in the end, when the communication and interaction flow involving different objects had to come together. On the final day of the workshop, all teams presented their demos, each in a separate booth, which was designed to resemble the context their designed for. All teams showed concise, focused demos, with working technology and a lot of attention to detail. They presented their concepts and design process to experts and received further feedback.

Conclusion

In this workshop, running for 9 days with 7 students teams, we started with the vision of “Awareness of Things” and designed fully experiential scenarios of how objects could collaborate and leverage their individual capabilities for joint goals. This was conceptually and technical uncharted territory for all of us. It was so impressive to watch the students engage not only with the topic and design challenge, but also with the technology that evolved throughout the workshop. It was amazing to feel a burst of creativity kick in after the very first days, as the teams gained confidence in their design skills. This coincided with the team members getting more and more comfortable with each other and seeing the benefits of different backgrounds and perspectives to design. This was, after all, part of the design challenge as well: understanding and leveraging the diverse strengths and approaches to design in a joint project. The teams learned to collaborate in new ways, on an unusual design challenge, with new technology and all that at very high pace. Everyone, students, teachers, and many supporters, helped to make this workshop not only a remarkable, but also extremely fun and rewarding experience, and I’m deeply grateful for that.

Japanese edition

The Japanese edition of “Coding Art” was published in June 2023 by BNN. It was masterfully translated by Tatsuo Sugimoto who we met in Sapporo in October this year. BNN also did a great job on the overall prpoduction; the book looks superb and feels great to work with.

Second edition

In the second edition, we have incorporated lots of feedback for text and code examples. But foremost, we have added two entirely new chapters on p5.js: starting with a few inspirations on generative art, we show you how to make use of the four steps and create a series of static and interactive pieces. And we do this with p5.js, so all pieces are ready for the web.

In the second new chapter, we take on a whole new challenge: creating experiences on mobile devices with acceleration and device orientation. Again we use p5.js and go through several steps. This is how you make generative art interactive.

Check out the new second edition and let us know about your feedback.

What are the three creative coding challenges that never get mentioned?

Ok, so you are interested in creative coding, visualization and generative art? Nice. What are the three most important challenges that only very few programs, books and tutorials address? First, the blank page problem, second, translating from idea to code, and finally, how to get out of feeling stuck. Recognizable? Absolutely, let’s dig in.

Blank page problem

Everyone experiences the blank page problem occasionally; it happens when you open a word processor, a blogging website, a new tweet, and also when you open your favorite coding environment. And then, despite your plan to create something, there is nothing. No words come out, no code flows from your fingers. It is as if our creative thoughts that were there just seconds ago, have vanished into the emptiness of the canvas.

So, what is really the problem here? We are motivated, we have the time, we have the right tools at hand–perhaps not even time pressure or an empty stomach (if not, solve those problems first). The problem is the empty canvas that has nothing that our creativity can latch onto. It is like trying to walk while floating in outer space, the motions of walking just don’t convert into momentum.

What can you do? Once you realize that the blank page inhibits you, that’s the first step. The next step is to realize that you were aiming too high for the first thing to come to your mind and that you need to feed your imagination somehow. In creative writing, there are techniques to get started, and same in creative coding: start with something really simple, a rectangle, a line or a circle. Put it out there and then another one. Soon, you see that the canvas is no longer empty, something has taken shape. Now, let your imagination slowly take over, move or remove something, add another thing. Arrange all elements, add chaos, delete everything except for the best elements, then start building again. Soon you will be on your way.

Why did I not suggest looking at examples and tutorials when suffering from a blank page? Very simple, “looking at” is almost as passive as staring at the blank canvas. What we need to do is to start making, start building confidence in what happens because we act. It’s not that we needed ideas, we needed a kick-off.

From idea to code

The second challenge, from idea to code, is related but slightly different: we struggle in turning an idea from our imagination into something visual, perhaps even moving and interactive. Your idea seems so clear, but there is no path to writing the code. You might even have sketched your idea on paper, and still… what you need is series of steps to make it. Doesn’t this feel familiar? Think of food, you crave that dish that you saw online or on TV, what do you need to make it? A recipe.

The world of creative coding is full of recipes. Every coding environment comes with examples and libraries, often a few clicks away. Open them and look for examples that look a little like your idea. Once you are reasonably close, start modifying the example, nudge it closer and closer to your idea. Soon you will have picked up the important steps of the recipe that you were missing before. Continue with this process, or even start from scratch, retracing the recipe towards your idea.

Feeling stuck

Finally, there is the challenge of feeling stuck. This is when the earlier momentum is lost, when we seem to go in circles, have too many options to choose from, or encounter a difficult technical challenge. This is difficult and easy at the same time: once we figure out why we feel stuck, getting unstuck is straight-forward. So, let’s focus on the why: in the first two cases, lost momentum and going in circles, we have moved fast for some time, outpacing our imagination. What you need is a break, to let your ideas catch up with the making again. In the second case, with too many options, we have lost the oversight and cannot see the best path forward. What you need is to zoom out and focus on the goal again. Give it a few minutes, then go through all options step by step to pick the one to continue with. You can always go back to the other options if your first pick does not work. Letting go of the pressure to make the right choice immediately is the key here. Experiment. The final case, a technical challenge, is common when starting with creative coding. So common that we have dedicated an entire part in our book to solving problems and pushing through the ceiling.

In the end, the creative coding journey is as much about what you create as what you learn about coding and creativity – and yourself. All three challenges above have to do with our own process, knowing what we need to do at which time. That is why creative coding is never just about a language, a tool or a few concepts, for most of us, we need to take an extra step to make the learning sustainable and self-driving: we need a process, a way of making with new technology that fuels itself and carries you over inevitable gaps of not-knowing and feeling stuck.

Now, what’s coming next for you? Check out our book website or dive into the book directly, – and show us what you make, give us feedback about what worked for you and how your own process looks like.

So, what happened instead? I will write a few things in the coming days and weeks about 2020. For the exact ETA of these writings I would like to invite you to read the first paragraph again. You see?

What if conferences would be virtual?

Background: In the context of the Corona crisis, many conferences and symposia might be canceled as a physical face-to-face conference and instead be held digitally as a virtual conference. This short document compiles a few ideas what organizers could do.

Despite the obvious disadvantages of not running a “physical” conference (presentation of demos, interactivity and art, F2F networking, random encounters, dinners and drinks, “live atmosphere”), there are benefits of running a “digital” conference: accessibility, sustainability, archiving communications, outreach, and diversity. Quit explicitly, conference costs and resources are not expected to change too much, as planning and operating a virtual conference is similar or more work and will cost a lot of money (compared to “free” Internet services). For the participating institutes and individuals, cost reductions (fuel, accommodation, food) and health benefits are expected (jetlag, travel risks, travel-related illnesses). A special mention here to accessibility: Conferences are difficult to navigate and consume – especially for impaired members of the community for whom getting to, standing in crowded ball rooms, listening to speakers, and viewing far-away slides is difficult. Conference mobility is a serious issue, the more the large the conference is. Yes, there are provisions, telepresence robots, mentors, helpers, volunteers, but this is seldom good enough for an equal conference experience and participation. Then: travel and budget restrictions, visa problems and so on.

Let’s focus on what might change to the better.

Sustainability: self-evident.

Quality of content: Given that the content of the scientific works (papers, posters, etc.) will be same as before, the focus should be on the quality of presentations. Perhaps a vetting process needs to be in place to ensure a great experience for the remote attendants. This is one of the aspects that make a virtual conference not really low-effort.

Interactivity: For a digital format, we will need to change how we interact in the context of a conference. Most interactions will, due to bandwidth limitations, be written and textual. This has benefits in understandability (no noise, more inclusive to non-native speakers) and also in following-up, referring-to and building of longer lasting conversations and hopefully collaborations. Presentations will need to change towards pre-recorded, more produced and choreographed moments that are possibly more polished and valuable to archive and preserve. At the same time, allowing for a variety of quality presentations can help speakers fit the presentation to their own preferences, abilities, and not least content.

Diversity, accessibility, …: All better than the current larger conference formats, perhaps on-par with current smaller conferences, not expected to be worse at all.

Networking: …is not everything. Although being high on the list of reasons to have and attend conferences, networking is only a side-effect of great content, effective communication, and proximity–all of which can be replicated online. An open and well-moderated “digital” conference format can help lower barriers for interaction especially for younger members of the community and allow them to get access to mentoring, advice, and new ideas.

Outreach, dissemination: Conference talks on YouTube, LinkedIn, Instagram, TikTok, other social media? No problem. The presence of scientific content on social media is probably a societal good and will be more important in the future. In other words, academia is lagging behind current publishing trends and it is time to catch up with how content is consumed.

How to do it?

Technology: The obvious point here is digital communications technology that is prevalent in the current self-isolating times. There are surprisingly few tailored and directly applicable solutions to running a large, highly interactive conference, but it is not without hope. Two possibilities that I see are either a fully hosted off-the-shelf business conference solution, or a tailored solution that patches different services together under a common, well-designed roof. For (1), managed solutions like WorkCast or similar might be suitable, for (2), a combination of YouTube live-streaming and Slack or other communication tools are possible. The choice depends on requirements, size, duration, and budget.

Operations: A digital conference needs lots of preparation and live action in terms of content moderation, speaker and participant facilitation, tech and production support. This will cost (time, money, nerves, …), but it is worth spending the effort.

Policy: Clear name policy, no pseudonyms, open communications, and well-maintained information channels during the conference.

What could be beyond the normal experience?

Time zones are a problem with respect to traveling; for a virtual conference, participants residing in diverse time zones can seemingly be a tough problem. However, it might also be a possibility to run the conference as a 48-hour single-track marathon, or a longer (and spaced) or shorter (non-stop) multi-track experience. Depending on the size of the community and divergence of interests in the talks, the former or latter might be more appropriate–it is a balance between choice and presentation hopping, and a more immersive focused conference experience.

Talks and all presentations are recorded, including the discussions. So, anyone involved in organizing the conference could experience everything and not just the parts they were free to attend. All recordings should be open on a platform and archived for the next years. This is not only good for reference, but also for outreach and possibly for teaching: instead of letting students just read a paper, they could start with the conference presentation and have a much more inspired reading experience.

Panels could be organized more ad-hoc and with larger participation of experts (who are not part of the conference even)

Inter-locality: While the conference is a virtual experience, participants could provide aspects of locality with live or filmed lab tours, studio sessions and in-situ show-and-tell, that might be a much richer perspective on design research practice and emerging technology than what was possible before. We can show the spaces where we work every day and inspire others. There could be a format for field-recordings of contexts that are interesting to design for and generally more engagement with linked materials that supplement a talk or presentation.

Virtual poster sessions could become more social, interactive, and useful as short presentations in a posters track. Demos could involve prototypes being shown in the studio environment, with good lighting and low noise. Demos and prototype sessions could make use of linked materials to provide details of the prototype itself and focus on the process of making and researching. Similar for the art track.

Plenary events (keynotes, town halls, introduction, closing) could be more interactive, public to guests and start every conference day with short keynotes. Award ceremonies could be locally produced or completely scrapped.

Virtual workshops could organize around sessions of papers and use materials that are sent out with postal mail or locally produced beforehand. They could have the format of a break-out session after a series of talks to turn the ideas into creative momentum and bring together the local research groups around selected topics of the conference.

Local chapters could participate together (by state, community, country, continent). This could be organized in universities, bringing together a larger local community than possible before in a conference context.

Networking activities, dinners and drinks could be hosted virtually with enough time available to chat and break out into smaller groups. This could be used for doctoral consortium-type feedback sessions (think of 10-people tables at a wedding).

From Wordpress to Jekyll

I should have researched tools to help the content transition first. But I guess, I went for an explorative approach to see what Jekyll could actually do. Technology assessment, but going all in.

While Wordpress as a collection of PHP scripts works as an installation on a web server, Jekyll runs on my local computer and renders all files which are then uploaded to the web server. Wordpress is always there and, after logging in, I can create, edit and publish pages and posts. There is a huge amount of plugins to handle things like comments (never needed), publications (even wrote my own plugin) and lots of other things. This is convenient, but having such an installation live on a web server also means that it might get hacked or abused in other ways. Static site do nothing like this, there are not executable parts. Static sites are just files on a web server that are served to a browser, which is very fast.

Jekyll

The first thing to get started with Jekyll is installing the rendering system on my computer. I remember that I tried to install Jekyll earlier, but failed initially and quickly gave up. I’m still surprised how unintuitive the first steps are (update Ruby, install brew, …). Fast-forward a few coffees later. Once everything was running fine with an example site, I started to play with the functionality which tightly coupled to the local file and folder structure. This a common approach to many framework-like systems that have strong conventions and provide good defaults, if one chooses to obey the conventions to the letter. What people often overlook is the effort in learning about such conventions and structures, about the implicit assumptions that the framework creators had when building the system over time. With Jekyll (and other similar tools) this was relatively straight-forward.

I started with the minima layout and adjusted it towards the original layout and style of my existing Wordpress site. This involved copying a few stylesheets and moving them to SASS, a stylesheet pre-processor that allows to compile a single final stylesheet from different source (and variables, and more expressive syntax than plain CSS).

From Wordpress to Jekyll, now really

After the first parts of the website were ready, started to look for ways to transfer the content from the Wordpress site. Obviously, there is a Wordpress plugin for that. Not exactly for the latest version, but still good enough. After a backup of my Wordpress installation, just in case, the plugin quickly rendered a full Markdown-formatted export of all posts, pages and images. Zipped up, ready to go.

The export was very helpful to just drop the text content into my new Jekyll site folder. The linked images needed a bit of adjusting and relative linking. Essentially, global find-replace and some folder moving.

Colors and styles

I still like the blocky style of the website, but everything was soooo grey (could have been the Winter as well). So, I added color gradients to the main background blocks and the corresponding navigation buttons. In the past, one would have probably used background images, but it is quite easy to use CSS color gradients today. So I did that.

There are some more smaller changes to the layout and typography. For example, I incorporated the footer from the minima theme and adjusted its style to the existing typography. Also, the header (and the entire site) is now responsive. This was also something where SASS helped: my previous site had basically larger blocks of CSS copied and adjusted for different screen widths (via media queries). With SASS I could streamline this.

Speed

The previous site running on Wordpress was a bit unpredictable in the way how different plugins would add scripts and stylesheets to the HTML document. Of course, there are ways to compress this, but I never got around to it. The resulting site performance was not really good. Several files, smaller and bigger, had to be transferred to fully render a page. Typically around 200KB, which is not tragic, but too much in relation to the content. The new Jekyll site is much leaner, around 50KB. In addition to being smaller in size, all files are static and be transferred directly from the server. Wordpress sites require rendering of files, which can be improved by using a cache. Nevertheless, rendering requires access to database, the PHP scripting engine and all this takes time and power.

Now, it’s all good. The end.

Ok, this blog post would not be complete without the following picture; I guess I’m guilty.