Codecamp 2024 summary

Posted on: January 14, 2024

---

We have a habit of starting a new year with a Codecamp with my friend Jarmo Hämäläinen. This year was no exception. Intensive coding from Thursday to Sunday. We used over 50 hours per person making it a total of over 100 hours of coding in just a few days.

As always, we had a lot of fun and learned a lot of new things. During the Codecamp you also always have struggles and problems but that’s part of the learning process. It is sometimes hard to explain all the things we did but let me try.

This year we expanded Jarmo’s existing Unity examples with different AI capabilities from Azure AI services portfolio. You can find more existing videos from Jarmo’s LinkedIn posts.

Our Avatar City is a virtual world where you can walk around and talk with different AI enhanced avatars. Avatars themselves are also walking around freely in the world. You experience it with VR headset and hand controllers to make it more immersive.

Experience is not scripted or pre-recorded, so you can talk with them about anything you want.

We’re using Finnish language because finally AI has reached the level that we don’t have to rely on English language anymore.

Our world has a lot of different characters, but let me introduce a few of them to you:

Noora is an artist. She sells her art.

Selma sells various antique items. She tells you about the items and their history.

Harri is a busy business guy, and he is pretty grumpy.

Talyn is from Mars, and he is quite an interesting character!

---

Technically our solution can be illustrated with the following diagrams from client and server perspectives.

Here is client-side flow:

flowchart TD A[Client] --> |Walk in the world
Find Avatar you want to talk with
Start talking| B(Avatar) B --> |Speech audio is collected

After pause in speaking| C(Audio clip) C --> |Send audio to server| D("Server
(see below flow)") D --> |Server responds| E{Contains
image?} E --> |No| G(Play response audio) E --> |Yes| F(Show image) F --> G G --> |Continue| B

Here is server-side flow:

flowchart TD A[Server receives request] --> |Use avatars parameters| B(Convert speech to text) B --> C{Image
required?} C -->|Yes| D(Generate image) C --> |No|E(Generate response text) D --> E E --> F(Convert text to speech) F --> End(Return response to client)

Here are a few images and links to videos:

Talking with Noora:

Talking with Talyn:

Talking with Selma:

Videos have been recorded from VR headset and you can see them from Jarmo’s YouTube channel. Here’s the summary video:

You can follow him in the LinkedIn to get notified when he posts new videos.

---

We had a lot of fun at our Codecamp.

I hope you enjoyed reading about our experience!