Image: A Culture ship descends in the cover art for Culture Unveiled

Space opera concept album brought to life with genAI​

As someone who has been experimenting with generative AI, I have been very interested in what's out there for audio generation. We have LLMs for written text, Stable Diffusion and Dalle for images, and while we have plenty of speech generation models, music generation has been a bit more obscure.

A couple commercial products are available, and I recently began experimenting with one of them, called Udio, and I have been having so much fun and want to share a weekend project that I put together using it.

The result is Culture Unveiled: Under The Wing, a 40-minute, 10 track album which serves as a spiritual cousin to Iain M. Banks' revered Culture series. It combines a variety of musical genres to explore the profound themes of discovery, transformation, and unity found in the Culture series. Each track is designed to take listeners on a journey through the narrative arc of Earth's integration into the interstellar community under the protective and enlightening influence of the Culture.

Iain M. Bank's Culture series introduces us to an advanced civilization where super-intelligent machines, known as Minds, oversee a utopian society. It's a world of post-scarcity where humans indulge in the freedoms that only superior technology can provide. While parts of Bank's novel State of the Art are set on Earth during the 1970s, Culture agents don't reveal themselves during that time, and leave our less advanced civilization to progress unhindered.

In Culture Unveiled, things have changed. Humanity is under threat from another alien civilization, one of the Culture's peers, and so they have revealed themselves in order to take humanity under the wing and protect them from the existential threat.

My aim here is to both show the current state of the art of these generative AI tools, and to engage in a bit of memetic engineering of our own culture. These generative AI tools have caused a massive immune response among certain segments of the populace due to their impact on jobs, energy use, and other ethical concerns, and I've been holding on to Bank's series as an alternative utopian vision. We are in for a wild ride in these coming years, and society needs to prepare for the shifts that these generative AI tools will bring. This not only showcases the creative potential of these technologies but also serves as a commentary on the transformative impact of AI on our society.

Technical Stuff​

Udio's audio model is capable of generating 30 second music clips based on genre and style tags, and can generate or use custom lyrics. By default, Udio generates two outputs each run, and you can extend tracks an additional 30 seconds at a time. So our basic workflow here was to enter our prompts, pick our favorite of the two, and then extend it. And so on and so on. While not a hard constraint, I did try to limit myself to picking two of these extensions for each track, which would wind up with names like 'Dawn of Gaia ext v1.2.2.1.1.1.1.1' or 'Synthetic Awakening ext v1.2.1.1.2.2.2.1' before I finished them. I also relentlessly deleted ancestors to make sure I moved forward with the project, which began in fits and starts a few weeks ago and culminated in a 12-hour Sunday session in which we did most of the generations.

The plot, track listing, lyrics and thematic prompts were all generated using ChatGPT. Most of what we did, engineering wise, was extend our preferred generations with our custom prompts and lyrics for each section. Then it was a matter or rolling the dice and waiting for the results. Udio has various knobs you can dial in for coherence, context windows, and other factors, which we wound up abusing liberally. Our prompts changed a great deal from section to section, which is how we managed some of the stark genre transitions that are in the album. Very rarely did we have to throw anything out and go back. There are no external edits, everything here was generated using Udio's user interface. I can only imagine the power of a similar pipeline that can leverage APIs and production quality audio engineering tools.

Image: A Culture ship descends towards Earth

Future Shock​

Let me say that Udio's audio engine is really good. Sure, there are some artifacts (happy accidents) that I left in, but I am really happy with how this turned out. I had so much fun doing this project, it was an extremely addictive experience. Right now it takes less than two minutes to generate 30 seconds of audio, but I am sure that in a few months they'll be doing two minutes of audio in 30 seconds. I am already imagining a future where music can be generated in real time, based on users' preferences. Albums on demand.

As an amateur musician, this technology is amazing. Udio's model can create basically anything you can imagine, and it and it's competitors are going to change how music is generated and consumed in the future. I spent a great deal of ~~~effort~~~ time crafting these tunes, listening to each generation, often from start to finish, just to choose which of the two was more interesting. And practically everything Udio does is interesting. It pained me to choose between certain versions of the output. The stereo outputs are so layered and are unconstrained with the actual limits of instrumentation or human dexterity. What starts as a simple guitar arpeggio on Eclipse of Shadows becomes it's own counterpoint, and builds until the brutality of the impending enemy is unleashed as a pair of thrash metal guitars. The sax solo in Cosmic Diplomacy is virtuosic, playing over a polyrhythmic bass and drums, and cacophony of piano chords. There were certain sections of songs that made me laugh or grin uncontrollably when I first heard them.

Udio's lyric modeling is still a bit hit or miss. The model was prone to ignore them some times, and getting it to stick to the script that I wanted was sometimes difficult. Getting rhythm and meter right was easier with the repetitive dance tracks, and hardest during the more new-age synth tracks. A bit more polish on the syllabic pronunciation is probably needed, but honestly I enjoyed the wrongness of it all, the disorientation into this liminal space of sound. And indeed, part of my goal with this project is to explore the Culture Shock that we are going to experience as these generative AI tools become part of our world.

Plucked from yesterday, Into a tomorrow we can scarcely imagine, The fabric of society, Unraveled and rewoven before our eyes.

New voices echo in the market square, Ideas once foreign, now laid bare. Machines that think, and ships that soar, A world unmade, to be reborn.

-Culture Shock

Introduction to COAT​

Since December of last year, we've been using ChainLit and OAI's Assistants API to build our LLM-enabled AiFGE app. Given the anticipated demand for boutique app studios in the coming months, we created a template version for easy deployment and scaling for clients. We've rebuilt the repository from scratch and made it public on GitHub.

ChainLit offers an excellent backend/frontend system for managing user and AI agent interactions. It handles user sessions with various OAuth providers and integrates with several popular model providers. Essentially, it's the closest thing to a hostable ChatGPT application we've found. Being model-agnostic, ChainLit allows you to build a local, self-sovereign Agent with thread history, file upload, and support for multiple users. Additionally, the ChainLit team developed Literal AI, a persistence, observability, and feedback platform integrated directly into ChainLit. They have also released an open-source, self-hostable data layer.

While locally hosted, on-device models might become the norm, currently, OAI is the best option for small teams. Since we haven't found Product-Market Fit (PMF) yet, it's not sensible to worry about the expense of hosting GPU/TPU time for any FOSS model we can deploy. Token costs are low, and we can always pivot later.

We decided to deploy our apps on Google's Cloud Platform as Docker-based CloudRun images, using shared Terraform state to manage our staging and production deployments. These scripts are included as well.

Technical Insights​

Over the past year, we've struggled to find a suitable hosting platform for agent configurations and deployments. Sure, there's LangChain, LlamaIndex, and frameworks like Haystack and AutoGen that might fit the bill, but a year ago, there weren't many clear standout options for scalable application delivery. Until November, we were using Steamship's agent deployment platform to interact with OAI's Chat API, but upon seeing the Assistants beta release, we switched completely. I am a glutton for punishment, apparently, but that's the cost of being at the forefront of frontier technologies.

OAI Assistants​

The OAI Assistants beta's killer feature for me was the integrated file search. Having all the file ingestion, embedding, and vector database details abstracted away was very attractive initially, but it has become a bit of a type-driven challenge as we've dealt with errors, missing data, and general issues while incorporating proper annotations and citations into the RAG-results. The initial implementation was somewhat passable, but keeping in mind that this tech is only going to get better, the V2 release data stores will be very interesting to explore. While the V1 retrieval search was limited to 10 files, the V2 allows up to 10,000.

The Assistants API is also interesting because it decouples the agent from the context of a thread. A quick recap on the evolution of OpenAI's endpoints:

• Completions: A one-shot, stateless call. For a conversation app, you needed to resend the entire message history for each call, which inflated the token count and ran up against the limited window sizes on GPT-3 (recall those measly 4000 token limits only a year ago?).
• Chat: This is the current model that I imagine most people are using. I believe this API is available for Azure customers and what most enterprises are building on right now, sans Google Gemini or whatever Amazon calls theirs. This API still requires devs to prompt the model and has some tool/function capabilities, but it abstracts the management of the conversation window, aka threads, from the developer, who can focus on prompt engineering and front-end work. RAG-pipelines still need an external provider.
• Assistants: In addition to managing the thread, the Assistants endpoint allows you to save agent configurations, each with their own instructions and defined tools, including file retrieval, code completion, and functions. It also lets you call different agents into the same thread, enabling some interesting use cases in multi-agent workflows, which we are just starting to explore. (See AgencySwarm for my current favorite implementation of this.)

And a note about GPT-o: if you don't think we're going to incorporate all those multimodal capabilities in the coming months, then just wait.

ChainLit​

So, on to the frontend. As someone who's been scared off of FE work since the great Web Standards wars of yore, finding an off-the-shelf front-end system for AI agents was a priority. That's one reason I was building on Discord. We did a little bit of testing with StreamLit examples but quickly saw that ChainLit was the way to go for production deployments.

• Websocket React client components available for custom frontend
• Bundled React Frontend
• REST API (FastAPI)
• Authentication (OAuth, custom)
• Websocket Session Handling (Starlette)
• Data Persistence (Literal, self-hosted custom)
• Discord and Slack integrations

Understanding ChainLit's event wrappers and context decorators, how threads, messages, and elements (attachments), and run steps (agent invocations) are sent to the UX and updated is crucial. I'm not going to get into the how-to of it here, but I have learned a lot from working through their source code to understand how the framework operates.

ChainLit has a wonderful cookbook that this repo started from, using their example of the Assistants code. We've spent the last two weeks converting it from the poll-driven function to an event-driven one that streams its response. We decided this would be a good milestone to open up the repo and give something back to the ChainLit community, and yes, to show off our work, warts and all.

I am releasing COAT for demonstration purposes, not because I think it's perfect or finished, but because it is a work in progress, and I feel the need to learn and build in public. We have a lot of things we want to do with the Assistants API, and part of our purpose here is recruitment, or 'finding the others' interested in helping out with our little workshop studio project.

Cheers!

Future Visions​

A quick aside about the debate ongoing around AI-doomers vs e/acc-Altman types: I'm fully open to criticism of power usage, resource constraints, bias, mass unemployment, ethical concerns around the training process, and general chaos and disruption that the next 3-5 years are going to bring, but The Times They Are A'Changing and I honestly don't think there's a way to turn it around without AI. My current take is that humans are so driven by the systems that we create around us, consciously or not (h/t Atomic Habits, Principles), that having a 'nudge' in the right direction from an intelligent system is too attractive to pass up. 'Right' in this case might be in the eyes of the beholder, but it is apparent that what our genes and evolutionary inheritance want from us short-term and what is best for us long-term are at odds, and we need some sort of interrupt. My Whoop fitness tracker is the ne plus ultra example right now. Of course, I don't want to see OAI or the other 'bigs' win at the cost of more corporate personhood and environmental degradation, but for better or worse, I think the only way out of that future is through it, and working like hell to be an optimist. These tools are knowledge-generating things, not just intelligent word calculators, and I think the flywheel of technological advancement still has a lot of kinetic energy in it right now. Try and keep up.

But that topic is for another blog post, and part of an AI-ethics project we are putting together. Today we are just glad to announce this release, hoping it might be helpful for others trying to learn, understand, and build agentic workflow systems. Feel free to reach out with any comments.

Right now, you should be able to fork the project and spin it up for a quick demo. We're hoping that anyone else building on Assistants will help build out additional features. Our current backlog, beyond the public issue board, is to build a multi-assistant system, where each assistant has workflow-based tools, tied to whatever API we can dream up. I would personally like a setup that can manage my grocery orders for me, but figuring out strategies for incorporating my various libraries of personal writings and downloaded knowledge into a coherent system is what's really driving me these days. I am also very focused on meta-tools that the agents can use to modify source code (Copilot is nice, and Workspaces looks even better); getting tools to the Assistants that will allow them to manage user context, issue boards, and save messages and entire threads to its data stores; building ingestion pipelines to learn books, PDFs, URLs, audio, video, anything. And then figuring out how to delegate and manage all these tasks and workflows in a way that we can keep the human in the loop.

The future is going to be mighty interesting, indeed.

Wade.Digital Update: Navigating Opportunities and Innovation in Q2 2024​

Greetings, Wade.Digital community,

As we transition deeper into the second quarter of 2024, it's an opportune moment to reflect on our recent activities and chart our path forward. The landscape of blockchain and AI continues to evolve, and so does our journey at Wade.Digital, amid successes and continuous learning.

Key Developments and Strategic Reflections​

VACOTechSprint and Partner Collaboration: Our participation in the VACOTechSprint, despite its ups and downs, has set the stage for new opportunities. We are still finalizing follow-ups and eagerly awaiting the public announcement of the winners. On a brighter note, our collaboration with [x] has flourished. We've made significant progress on the AiFGE app, which stands out as a highlight of this period. Our discussions with [x] have opened doors to potentially take on a role within their community management or developer relations teams, fostering further collaboration.

Crypto Market and Treasury Insights: The crypto market has treated us well, with Bitcoin maintaining its rally and our strategic trades keeping us within our risk tolerance. Our treasury management strategies, including the recent conversions from USDC to USD and the clearing of less favorable positions, are reflective of our cautious yet proactive approach to finance.

Strategic Vision Reevaluation: Recently, I revisited our strategic vision for 2024. The aim is to position Wade.Digital as a leader in leveraging advanced technology for community engagement and operational excellence. However, it's clear that while technology and strategy are our strong suits, the aspect of community engagement needs reinvigoration and clearer integration into our operations.

Looking Ahead: Operational Focus and Community Building​

AI and Automation: We continue to refine our AI tools, with significant strides in the AIFGE project. Our partnership with [x] may also soon yield a new role, helping us bridge community engagement and development efforts.

Content Creation and Community Outreach: In our quest to enhance community ties, we've begun creating valuable content that reflects our development processes and insights. This includes a series of YouTube videos aimed at building a long-term community, though admittedly, this is a work in progress that will evolve with us.

Financial Strategy and Sustainability: As stewards of Wade.Digital's resources, we are mindful of our financial strategies, especially in maintaining the balance between asset retention and operational liquidity. The market's volatility necessitates a conservative approach to asset management, aligning with our vision for sustainable growth.

Reflecting on Our Pillars of Strategy: The five pillars of our strategy—discovery, building, decision-making, deployment, and user engagement—remain central to our operational philosophy. Each pillar is being examined and refined to ensure it supports our broader mission of making science fiction a science fact.

In conclusion, while the paths we navigate are complex, our commitment to innovation, strategic integrity, and community engagement remains steadfast. We look forward to what the rest of 2024 holds and are excited to continue this journey with all of you.

Stay tuned for further updates as we progress through the year and refine our strategies to not just meet but exceed our expectations.

Warm regards,

Michael Wade

Founder, Wade.Digital

We have released v0.1 of our YouTube Video Pipeline repo.

Demo Video​

Use and Purpose​

This repo is a snapshot of the current workflow we're using to publish impromptu videos to our YouTube channel as part of an exercise in discipline. It's called 100 Days Challenge, and is mainly an excuse for me to spitball topics that I want to talk about. I'm trying to make the process as frictionless as possible, and this is the result.

The current constraints that we have for the project is that we'll record a short video, about 10-15 minutes on my iPhone, and upload it to YouTube without any edits. I am however, using a transcript of the video, fed through an LLM, to provide the title, summary, and other metadata about the video.

This repo relies on a separate WhisperASR service to do the transcription, but you can probably use whatever third-party transcription engine you want.

Workflow​

I had originally been doing the process via a kludge of: Airdrop from my iPhone to Macbook; the Obsidian transcription plugin; an ASR-endpoint in my lab; ChatGPT; and the YouTube web app. I would like to automate the entire process. So far we've writen automator files to strip the audio before passing it to the ASR (it's less bandwidth intensive to strip it) and returning the transcript file. From there we swapped out ChatGPT for an API call using CrewAI, which is the latest Agent-Task management tool we've been playing with. We've also added a python-upload CLI tool, written in Python, that we've hacked into today.

There is still a lot of room for improvement here, and we'll be making additional tweaks as we continue to use it over the next eighty days or so. We're releasing it here as-is, and have no plans to support it or maintain it at all.

We will be prioritizing any friction we run into with the model, and will likely be making improvements to the CrewAI loop to make sure it's passing the proper data to the YouTubeUploadTool, so that we might make it a proper tool which we can fork back to CrewAI.

In the digital era, where technology evolves at lightning speed, staying ahead of the curve is not just an advantage—it's imperative for success. Among the various skills crucial for navigating this landscape, AI literacy stands out as a fundamental necessity.

Exciting News from Wade.Digital!​

A recent LinkedIn study shed light on a pressing issue: the AI literacy gap in the workforce. The study revealed that merely a third to half of UK companies are actively fostering AI literacy among their employees. This gap poses significant competitive challenges but also opens up a realm of opportunities for those ready to embrace change.

At Wade.Digital, we're seizing this opportunity to make a difference, and would like to invite you to sign up for our upcoming content dedicated to enhancing AI literacy. Our mission is to equip professionals and aspiring talents with the prowess to harness AI tools for research, data analysis, customer service automation, predictive maintenance, and beyond.

Our course is meticulously crafted to transcend traditional learning boundaries. It's not just about understanding AI; it's about embedding a culture of continuous learning, ethical AI practices, and interdisciplinary collaboration within your organization. We aim to transform AI integration from a mere operational tool to a cornerstone of innovation and competitive advantage.

As we gear up for the launch, we invite you to join us on this transformative journey. Whether you're looking to future-proof your career, lead your organization into a new era of innovation, or navigate the ethical dimensions of AI, Wade.Digital is your compass.

Stay tuned and sign up for exclusive previews, insightful updates, and more as we embark on this exciting venture together. Let's close the AI literacy gap and pave the way for a more innovative, adaptable, and ethically conscious digital future.

---

Introduction to AI Literacy: Empowering the Future Workforce​

This course is designed to equip participants with foundational AI literacy, focusing on practical applications and ethical considerations in the digital age. Through engaging content and hands-on projects, learners will explore how AI can be leveraged across various domains, including research, data analysis, customer service, and predictive maintenance, fostering a culture of continuous learning and innovation.

Course Modules:​

Week 1: Introduction to AI and its Impact​

• Overview of Artificial Intelligence
• History and Evolution of AI
• AI in Today's World: Applications and Case Studies
• Ethical Considerations and Responsible AI

Week 2: AI for Research​

• Leveraging AI for Enhanced Research Capabilities
• AI Tools and Techniques in Research
• Case Studies: AI-driven Breakthroughs in Research

Week 3: AI in Data Analysis​

• Fundamentals of AI in Data Analytics
• Tools and Platforms for AI-driven Data Analysis
• Real-world Applications and Success Stories

Week 4: Customer Service Automation with AI​

• Introduction to AI in Customer Service
• Chatbots and Virtual Assistants: Design and Deployment
• Enhancing Customer Experience with AI

Week 5: Predictive Maintenance through AI​

• Predictive Maintenance: Concepts and Importance
• AI Techniques in Predictive Maintenance
• Implementing AI for Operational Efficiency and Cost Reduction

Week 6: Beyond the Basics: Advanced Applications and Future Trends​

• Exploring Advanced AI Applications
• The Future of AI and Emerging Technologies
• Building a Culture of Continuous AI Learning in Organizations

Week 7: Ethical AI and Sustainable Practices​

• Understanding Ethical AI and Its Importance
• Guidelines and Best Practices for Ethical AI Use
• Case Studies: Ethical Dilemmas and Solutions in AI

Week 8: Capstone Project​

• Applying learned concepts to a real-world problem
• Group projects to foster collaboration and application of AI literacy
• Presentation and feedback session

Learning Outcomes:​

By the end of this course, participants will be able to:

• Understand the foundational principles and applications of AI across various domains.
• Apply AI tools and techniques for research, data analysis, and customer service.
• Implement AI strategies for predictive maintenance and operational efficiencies.
• Navigate the ethical considerations in AI applications and advocate for responsible AI use.
• Foster a mindset of continuous learning and adaptability in the face of evolving AI technologies.

Recommended Prerequisites:​

This course is designed for beginners with no prior experience in AI. Basic computer literacy and an interest in technology and innovation are recommended to maximize learning outcomes.

This syllabus provides a structured framework for your AI literacy course, blending theoretical knowledge with practical applications and ethical considerations to prepare participants for the AI-driven future.

Welcome to the Future: Introducing Wade.Digital​

Greetings, Innovators and Dreamers!​

Today marks the beginning of an exhilarating journey into the uncharted territories of blockchain and artificial intelligence — welcome to Wade.Digital! Our mission is clear yet audacious: to turn the fantastical elements of science fiction into the tangible realities of our daily lives. At Wade.Digital, we're not just observers of the digital revolution; we are active participants, builders, and visionaries shaping the future.

Our Philosophy: Building Tomorrow, Today​

Wade.Digital is more than a company; it's a movement. Inspired by the boundless possibilities of technology, we've embarked on a quest to design a future where digital innovation enhances every aspect of human life. From streamlining mundane tasks with AI to creating decentralized communities with blockchain, we're laying the foundation for a world where technology serves humanity, not the other way around.

What We Do: Innovation at the Intersection​

At the core of Wade.Digital lies a commitment to innovation, particularly at the intersection of blockchain and AI. Our projects range from developing AI-driven tools that simplify complex tasks to exploring blockchain's potential to foster transparent, equitable communities. We're here to build systems that empower individuals to design their lives with technology, fostering personal growth and professional development.

Join the Community: Collaboration is Key​

One of the pillars of Wade.Digital is community. We believe in the power of collaboration, shared knowledge, and collective creativity. Our blog will serve as a platform for not just sharing our advancements and insights but also for engaging with fellow innovators, tech enthusiasts, and anyone curious about the future of technology. We encourage you to join the conversation, share your ideas, and collaborate with us on projects that matter.

Looking Ahead: The Journey is Just Beginning​

As we chart our course through the ever-evolving landscape of technology, our blog will be your window into the world of Wade.Digital. From deep dives into our latest projects to reflections on the broader implications of digital innovation, we'll be sharing it all right here.

We're excited to embark on this journey with you. Together, let's explore the possibilities, push the boundaries, and build a future where technology unlocks new horizons for humanity.

Welcome to Wade.Digital — where the future is not just imagined; it's created.

---

Stay tuned for updates, insights, and more as we navigate the exciting world of blockchain and AI. Subscribe to our newsletter, follow us on social media, and join us in building a future where technology and humanity converge in harmony. Welcome aboard!