In my last post, I had an existing web application that I wanted to add .NET Aspire to help with orchestrating external dependencies.

In particular, my web applicaiton is a LLM chat application that relies on Ollama for chat completions.

For all intents and purposes, Ollama is it's own microservice that my web application consumes and communicates with through it's REST API.

In this post, I show how I can add Ollama to my solution so that my web application has a reliable and consistent developer experience.

From Last Post

Here is a quick recap of the last post. If you don't need a refresher, skip to the next section.

To my existing ASP.NET MVC Web application solution, I:

• added a .NET Aspire AppHost project using the project template
• added a .NET Aspire ServiceDefaults project using the project template
• configured my web applicaiton to use builder.AddServiceDefaults();
• added http and https entries to my launchSettings.json file in web application

For specific details, please check out my last post here.

Manually Running Ollama

For chat completions, my web application relies on Ollama server running and configured to serve up a local LLM model.

In particular, my web application uses the OllamaApiClient type for the OllamaSharp library to communicate (see GitHub for it here) with an Ollama server over HTTP.

The code for creating the client is here.

IChatClient chatClient = new OllamaApiClient(new Uri("http://localhost:11435/"));
IChatClient summarizeClient = new OllamaApiClient(new Uri("http://localhost:11435/"));

For this web application, I use separate clients for chat completion and summarizing. The summaryizeClient is used to construct a title for new chat conversations.

Check out this other post of mine that discusses changing my web application from using the Microsoft.Extensions.AI.Ollama library to using OllamaSharp.

As you can see, an API client is instantiated for Ollama running at the following REST api address http://localhost:11435.

Note the default port for Ollama's REST API is 11434 but here I use 11435 for reasons you'll see shortly

To get this to work, I would have to ensure that the Ollama server is running on my machine and that the underlying model was pulled.

Sometimes, I'd forget and get a runtime error when trying to use my chat application.

The better way is to let .NET Aspire run Ollama for us.

Configure .NET Aspire to Run Ollama

With .NET Aspire part of my web application solution, I can add Ollama as a dependent resource as follows.

To my AppHost project, I add the CommunityToolkit.Aspire.Hosting.Ollama Nuget package.

In AppHost project, this is the current state of the Program.cs file.

In AppHost project, I can then modify the Program.cs file to add Ollama as a resource.

const int ollamaPort = 11435;
const string OllamaModelName = "llama3";

var ollama = builder.AddOllama("Ollama", ollamaPort)
    .WithDataVolume();

ollama.AddModel(OllamaModelName);

This code defines constants for a custom Ollama port and specific model that Ollama should use.

It then chains together a call to .AddOllama(...) extensions method (from the CommunityToolkit library) to add the resource with that custom port and data volume.

And lastly, the code calls .AddModel(...) to tell the Ollama resource to use llama3.

The AppHost Program.cs file looks like this.

You can now run the application and see that the Ollama server resource along with the llama3 model are available in the resource dashboard.

One last bit here, is to add a call to .WaitFor(...) on the ollama resource so that my web application will wait until Ollama is available before it starts up. This ensures that no race conditions happen where the my web application is ready but Ollama is still starting up in the backgroun.

builder.AddProject<Sidekicks_MultiApp_OCWeb>("SidekicksOCWeb")
    .WaitFor(ollama);

Next Steps

So far eveything looks good, but the application would fail if I deployed to server because my web application is hardcoded to use localhost when calling Ollama.

.NET Aspire provides a way to avoid hard coding this reference which I will post about next time.

Until then, keep coding.

.NET Aspire is pretty cool.

Riding the wave of AI hype, I decided to write my own AI Chat completion web application.

Microsoft did a great job documenting how you can get up and running with your own AI chat application in a few dozen lines of code.

Personally, though, I don't like feeding the beast of a cloud based LLM like OpenAI or Azure OpenAI because writing (to me) is actualized thinking and I'm not about to send my thoughts to a 3rd party.

So, long story, short, I wanted to run a local LLM where I know the messages I send to a chat completion services stay with me.

Local LLM Chat

Luckily, Microsoft is quick to understand this and provides great guidance on how to build a chat app with a local LLM.

The linked tutorial from Microsoft has you running a 3rd party thing called Ollama which provides a RESTful API to connect to a LLM running on your machine.

Ollama is a great tool for finding, downloading and running local LLM models.

They even have a docker container. And as long as it's running I can point my chat app to it and my chat mesages stay local.

Opinionated Orchestration

But writing software that depends on a separate 3rd party running service is a pain.

Enter .NET Aspire, a local dev orchestration framework for cloud native / microservice based systems to improve the developer experience.

In a way, you can think of Ollama as a microservice that my local chat application can utilize.

So adding .NET Aspire to my project is a natural next step.

It is super easy.

Implementation

Start by adding the .NET Aspire App Host project to your solution.

You'll see the AppHost project added to your solution with a Program.cs file you can use to start orchestrating parts of your app.

First, be sure to add a project reference to your web project in your AppHost project.

Open the Program.cs file in the AppHost project, and add a reference to your project.

In my chat app, SidekicksOCWeb is the project I want to start up with the .NET Aspire AppHost runs.

The fully qualified name is Sidekicks.MultiApp.OCWeb.

A cool thing that Visual Studio intellisense does is give you a type name for your project. My web project is named Sidekicks.MultiApp.OCWeb and .NET Aspire generates a type Sidekicks_MultiApp_OCWeb (as seen above) so it is easily referenced.

Here is the code for the Program.cs file of the AppHost project.

using Projects;

var builder = DistributedApplication.CreateBuilder(args);

builder.AddProject<Sidekicks_MultiApp_OCWeb>("SidekicksOCWeb");

var app = builder.Build();

await app.RunAsync();

And be sure to add the reference to your web project from your AppHost project.

Run it

Now is a good time to check that it's all working.

Set your start up project to be your AppHost project. One way to do this is to right-click your AppHost project and select select as startup. Another way is to select your AppHost project from the startup project dropdown.

Now go ahead and run your app.

The first thing you'll see is the .NET Aspire startup log console window.

You should also see the .NET Aspire Dashboard open in a browser as well.

The Resources tab shows you your web project and any other resources you have for your integrated system.

Go ahead and click through all the stuff in the .NET Aspire Dashboard.

The Console tab gives you logging from your web app.

The Structured, Traces and Metrics tabs look to be empty though. So let's go ahead and fix that.

.NET Aspire ServiceDefaults

Stop your application and add a new project to your solution.

Use the .NET Aspire ServiceDefaults project template.

You'll get a new project with a single static type called Extensions that you can use in your web application project.

Reference the ServiceDefaults project from your web application project.

Then in your Program.cs file, just add builder.AddServiceDefaults() right after your WebApplication.CreateBuilder(app) call like so.

var builder = WebApplication.CreateBuilder(args);
builder.AddServiceDefaults();

Go ahead an run your application now and see that Structured logs tab is now populating with information.

Out of the box!

Then, as you start sending web requests to your web applicaiton, the Traces tab will fill in with information about your web traffic calls.

In the Metrics tab, select your web project resource and see the rich information there as well.

Resource URLs

One last thing to do is to set the URL for your project resource. In my case it's SidekicksOCWeb.

See how the URLs column is empty.

I tried many different searches for "how to display my web project url in .NET Aspire dashboard" or ".NET Aspire dashboard missing web project's URL" but nothing came up.

Nevertheless, I was able to compare a working Sample to my application and figured out that I need to do something in my web application project to get the URLs to show up.

Navigate to you web projects Properties->launchSettings.json file, and add entries for http and https if you don't have them.

Mine originally looked like this.

And here are the missing entries for http and https, ensuring that the applicationUrl value is my web application's actual launch URLs.

    "http": {
     "commandName": "Project",
     "dotnetRunMessages": true,
     "launchBrowser": true,
     "applicationUrl": "http://localhost:5000",
     "environmentVariables": {
       "ASPNETCORE_ENVIRONMENT": "Development"
     }
   },
   "https": {
     "commandName": "Project",
     "dotnetRunMessages": true,
     "launchBrowser": true,
     "applicationUrl": "https://localhost:5001;http://localhost:5000",
     "environmentVariables": {
       "ASPNETCORE_ENVIRONMENT": "Development"
     }
   }

The updated launchSettings.json file looks like this.

After making that change, go ahead and run the solution and you'll see that the URLs are now correctly populated in the .NET Aspire Dashboard.

Recap, Next Steps

And that is all there is to adding .NET Aspire to your existing web application.

Next, I'll show how to add an Ollama resource to .NET Aspire and then call it from my existing web application.

This is where .NET Aspire really shines.

Stay tuned.

Things change in a matter of days or weeks not months.

As a pet project, I created my own chat assistant application utilizing Microsoft technologies and Ollama for local LLM model chats.

I created it a few months ago when a number of the nuget packages were in preview. Today I went to update the packages and most of several minor versions ahead.

I blindly updated to the latest versions of the packages and now I have build errors.

The first issue was with a project that references Microsoft.Extensions.AI.Ollama, specifically the 9.7.0-preview.1.25356.2 which is deprecated and Microsoft recommends using OllamaSharp as an alternative.

Went ahead and uninstalled Microsoft.Extensions.AI.Ollama package and installed OllamaSharp, but now I have build errors.

Checked the OllamaSharp documentation on GitHub here to see I needs to change a few things.

The client instantiation type changes from OllamaChatClient to OllamaApiClient.

 //IChatClient summarizeClient = new OllamaChatClient(new Uri("http://localhost:11435/"));
IChatClient summarizeClient = new OllamaApiClient(new Uri("http://localhost:11435/"));

Then, the method call to get the chat completion changed from CompleteAsync to `

//var name = await summarizeClient.CompleteAsync(summarizeHistory, summarizeChatOptions, cancellationToken: responseCancellationToken);
var name = await summarizeClient.GetResponseAsync(summarizeHistory, summarizeChatOptions, cancellationToken: responseCancellationToken);

The next compiler error is related to the Chat completion itself.

Fix by accessing the .Text property on the response itself rather than going through the .Message property.

 //currentConversation.Name = name.Message.Text;
 currentConversation.Name = name.Text;

Of course, the most important change being to the streaming responses itself to replace .CompleteStreamingAsync(...) method.

//await foreach (var item in chatClient.CompleteStreamingAsync(chatHistory, chatOptions, cancellationToken: responseCancellationToken))
await foreach (var item in chatClient.GetStreamingResponseAsync(chatHistory, chatOptions, cancellationToken: responseCancellationToken))

So far, so good - but the problem is with assign to the .Text property of the item in that list. For this particular implementation, it happens to be a ChatMessage type from the Microsoft.Extensions.AI.Abstractions library.

The .Text property used to be writable but now it is readonly.

Turns out that this code is a relic of a prior design that is no longer necessary. The only purpose for appending to the .Text property was so that I could use it to construct the MarkupString on the next line.

Analyzing the code even more, I am constant querying my dataStructures looking for the latest Assistant message to get the index simply to align with the index of the messageHistory array.

That is not necessary now. So let's refactor.

var lastMessage = messageHistory.LastOrDefault(x => x.Role == "assistant");
int index = messageHistory.IndexOf(lastMessage!);
string response = "";
await foreach (var item in chatClient.GetStreamingResponseAsync(chatHistory, chatOptions, cancellationToken: responseCancellationToken))
{
		response += item.Text;                
		messageHistory[index].Markup = new MarkupString(Markdig.Markdown.ToHtml(markdown: response, pipeline: MarkdownPipeline));

		this.StateHasChanged();

		if (responseCancellationToken.IsCancellationRequested)
		{
				break;
		}
}
chatHistory.Add(new ChatMessage(ChatRole.Assistant, response));

First, you'll see the index finding logic outside of the await foreach, and instead of using the chatHistory array, let's use the messageHistory array because that's what we care about updating.

Next, you'll see we use a simple string for concatenating the reponse which is then used to construct the MarkupString.

Lastly, the chatHistory array is updated with a new ChatMessage object after the reponse has completed streaming. This is what allows us to avoid trying to write to the ChatMessage's .Text property.

It is also more aligned with how Microsoft has documented that you add a ChatMessage to history.

After these changes, I was able to run the application and it worked as it did before.

Interestingly enough, there is a noticable speed up which may have been from avoiding the repeated steps to look up the index.

Until next time.

As a programmer, I have a surprising interest in infrstratucture and devops.

I learned Kubernetes because it seemed very cool.

I setup a homelab using VMWare because it also seemed cool, but having a giant rack running in my closet seems strange. Plus the hardware is aging fast and already quite old.

Enter Turing Pi.

Turing Pi Clusterboard

I think this was probably a targeted ad on social media, but it sure got me good.

The Turing Pi 2 is a mini clusterboard that allows you to add multiple computer modules to for a mini computing cluster.

What is a mini computing cluster?

My simplistic understanding is that you can configure your cluster in all sorts of ways, one of which is to configure each compute module as a node in a a kubernetes cluster.

So that's what I did.

Naked on the Desk

I've had my Turing Pi 2 board for a long time but I've only started working with it recently because of personal time scarcity.

For quite a while, it was siting on my desk without a case just waiting for someone or something to break it in a bad way.

Turing Pi Mini-ITX Case

I just recently received my mini-ITX case from TuringPi and it has rejuvenated my interest in getting this new kubernetes cluster up an running.

Check back for future updates to this article for more details on this build.

This is the short path you can take to becoming a software engineer.

The Meaning of the Phrase

It's important to ask yourself what you mean when you ask "how to become a software enginner" because different people may have different ideas about what that means.

I would argue that any person who can code a complete application qualifies as a software engineer.

Not all applications are equal, though.

Full Stack Application

Software comes in all shapes and sizes.

You have web apps, desktop apps, mobile apps, daemon/background apps, embedded apps, games, command line utlities, and more.

A "full stack developer" is one who is competent in all layers of a particular application stack, where "application stack" is loosely the different aspects of a software system. Namely: User Interface (UI) & User Experience (UX); core logic or business logic rules; and data persistence like a database.

That's a bit of an oversimplification but it covers the major areas necessary for a software system to provide some kind of value to some kind of users.

A person who can write code to deliver each part of the full stack of an application can certainly call themselves a software engineer.

One Approach

Here is a brief step by step of how one might become a software engineer, which is sort of a bare minimum that I'd look for what considering candidates for open software engineering positions at my company.

1. Create, Retrieve, Update & Delete (CRUD)

Create a web application that allows a user to create, retrieve, update and delete some kind of data. This is a classic "CRUD" application that covers basic functionality found in most software used by end users.

One example of this is a "todo" application that allows you to keep track of your todo list.

Another example is a movie database for you and/or your family to track movies that you own.

Once you've mastered creating a web application to store, retrieve and update data in a database, you're ready to then apply those skills to a specific domain skill.

2. Applied Skills

Software engineers and software developers are modern day blacksmiths.

We take code and forge into into valuable functionality that a specific person or group finds useful for their work.

In essence, we make software for other people who have problems that software/technology can solve.

A more complex project to work on might be a double entry bookkeeping system for you to track your finances.

This example requies you to understand how double entry bookkeeping works and that you build a software system conform to the rules and constraints of that particular process.

There are certain rules, or business rules, that a bookkeeper (one who does bookkeeping) must follow to keep a set of financial books for a business.

As a software engineer, your responsibility is to understand the "domain" of that bookkeeper, to understand what bookkeeping is, and to create a software system that allows the bookkeeper to do his/her job with your software system.

Once you've take your software skills and applied them to a specific domain skill, that is how you know you've definitely become a software engineer.

The next step is to understand how your software application is just one part of a greater system.

3. From Software to System

Where a software application allows a users to do his or her job, a software "system" encompasses how that software application existence with other systems in this world.

For example, to build on the prior mentioned bookkeeping application, you can imagine how that application can exist alongside other systems, like the bank account for the business.

Imagine refactoring your bookkeeping applicaiton to interoperate with that bank account so it can automatically read transactions to allow the bookkeeper to classify them.

To achieve that, you would have to know and understand how (if possible) you can integrate with the bank's system likely through some kind of Application Programming Interface (API) to read data.

Most times when you want to read data from another system, you'd follow some kind of developer guide provided by the bank to interface with their system.

Once you've added an integration like this, you're no longer creating just a software application, but you're maintaining a software system.

Trade Craft

Software Engineering is a unique combintation of science, art and applied social skills.

And it certainly is a craft that is ever evolving requiring software engineers to change with it.

Consistency is critical.

Continually writing code to make applications is important to improve your craft.

Understand how tha craft can be applied to other people's profession is next level and further increases your stance and a software engineer.

And then finally, once you understand how you can integrate your application with other systems, you're an advanced software engineer that can apply systems level thinking to solve real problems.

Good luck and keep coding!

I've never been to a technology conference before, and although Orchard Harvest is a relatively small conference compared to other open source projects, it is by no means a small experience.

The Conference

This year's Orchard Harvest was the first in-person event in several years. With a few dozen attendees, the conference is small by comparison to other major opensource projects.

Nonetheless, the people there, including core maintainers, business owners, public sector users, and other Orchard Core enthusiasts, brought with them incredible passion surrounding the small but mighty content management system (CMS) and application framework.

The Project

I found Orchard Core a couple years ago while researching the best way add mutlti-tenancy to my client's custom software reporting system.

I quickly realized the immense value Orchard Core offers out of the box in terms of features.

You get user management, content management, authentication/authorizations, workflow management, and of course multi-tenancy (plus many, many other features!)

Even more valuable is the ability for Orchard Core to be added to an existing ASP.NET MVC (or Razor Pages) web project simply by referencing a single nuget package.

Lastly, the open source GitHub repository for Orchard Core illustrated the active engagement of a passionate community of developers and users.

I read all the documentation I could on Orchard Core, watched all the YouTube tutorial videos I found (thank you Lombiq!) and set out to do something I'd never done before.

I set out to get involved and contribute to an open source project.

Open Source Development

The experience was daunting.

I've developed .NET software systems for over 20 years, almost as long as .NET has been around.

My programming confidence was always high when engaging within my own teams on private projects.

But open source was different.

I would be submitting code that the whole world would see. And I was scared that I might embarass myself.

It took a long time to submit my first pull request. My imposter syndrome was strong, but after several month I garnered enough courage to submit my first PR for Orchard Core.

And I'm glad I did.

I learned a lot about the process to get your PR approved. The need to iterate on the changes, and the necessity to keep your PR branch in sync and up to date with the target branch.

The Orchard Core community was very welcoming and provided constructive feedback.

The experience was very rewarding. Enough so that I was more than happy to hop on a plane and join the community at Orchard Harvest 2024.

What's Next

I'll continue contributing to the Orchard Core project where I can.

I personally think it is a wonderfully useful technology for any .NET developer out there.

Even if you don't plan to use Orchard Core CMS or application framework in your project, you should at least explore the code to get an idea of how ASP.NET can be used.

In the words of one of the core maintainers for Orchard Core, the code is beautiful.

I personally think they're somewhat magical.

I would love to "chat" with my own data to gain insights, etc.

But I don't want to share my data with anyone.

Enter local Retrieval Augmented Generation (RAG).

What is RAG?

Retrieval Augmented Generation is a way for you to include your own data when chating with a Large Language Model (LLM) chatbot.

Including your own data gives tremendous context to the LLM to provide more personalized, or domain specific information not otherwised found in the large corpus of text used to train these language models.

Additionally, you can have the chatbot provide citations references to your own documents your user's consideration.

What is Semantic Kernel?

Semantic Kernel is Microsoft's framework for orchestrating artificial intelligence (AI) services.

You can learn more by checking out my series on it:

• Part 1 - Getting Started here
• Part 2 - Planning, Memory & Embeddings here.

I used Semantic Kernel because I love programming in .NET and C#.

A Contrived Example

Here is a basic list of facts that will simulate "my data", which I'll use to then chat with a local LLM and see what I get.

Credit: This example was inspired by another's blog post, but I cannot find it. I'll update my post if I find the orginal article I used when doing this.

            var facts = new OrgFact[]
                {
                    new("Our headquarters is located in Sydney, Australia.", "Headquarters", "City: Sydney"),
                    new("We have been in business for 25 years.", "Years in Operation", "Years: 25"),
                    new("Our corporate sponsor is the Melbourne Football Club.", "Corporate Sponsorship", "Team: Melbourne Football Club"),
                    new("We have 2 major departments.", "Departments", "Number: 2"),
                    new("Our team includes developers among other professionals.", "Occupation", "Job Title: Developer"),
                    new("Our team enjoys outdoor activities such as bushwalking.", "Team Activities", "Activity: Bushwalking"),
                    new("We have a company pet policy that allows dogs.", "Company Pet Policy", "Type: Dog"),
                    new("We prefer catering options featuring Australian cuisine.", "Catering Preferences", "Cuisine: Australian"),
                    new("We have expanded our operations to 5 countries.", "International Presence", "Countries: 5"),
                    new("Our staff includes graduates from the University of Sydney.", "Education", "University: Sydney"),
                    new("Our team is multilingual, speaking 3 languages.", "Languages Spoken", "Number: 3"),
                    new("We have a strict allergen policy, including precautions for peanuts.", "Allergen Policy", "Allergen: Peanuts"),
                    new("We support athletic achievements, such as participating in marathons.", "Athletic Support", "Event: Marathon"),
                    new("We have a company-wide collection of Australian art.", "Company Initiatives", "Item: Australian Art"),
                    new("Our team enjoys the Australian spring season for company events.", "Seasonal Preferences", "Season: Spring"),
                    new("Our corporate book club's favorite book is 'The Book Thief'.", "Corporate Book Club", "Book: The Book Thief"),
                    new("We offer vegetarian, vegan, gluten free and halal options in our corporate diet policy.", "Dietary Policies", "Diet: Vegetarian"),
                    new("We actively support volunteering in local community projects.", "Community Engagement", "Place: Local Community Projects"),
                    new("We aim to expand our presence to every continent.", "Expansion Goals", "Goal: Every Continent"),
                    new("Many of our staff members hold advanced degrees, including in Computer Science.", "Advanced Education", "Degree: Master's in Computer Science")
                };

When loaded into my local chat bot, I get the following results:

Conclusion

Retrieval Augemented Generation is a great way to chat with your data.

And Semantic Kernel is a C# developer's way to create some great AI experiences.

This is Part 2 of a multi-part review of Microsoft's Semantic Kernel SDK.

You can read Part 1 - Getting Started here.

Intro

In Part 1, we covered getting started with Microsoft's Semantic Kernel by working through the Polyglot Notebooks in the GitHub repository.

We covered tools used, contstructed a Kernel, creating prompts with arguments, and a simple chat bot example.

This part continues working through the Polyglot notebooks to cover planning, memory, embeddings and other features.

Planning

The concept of Planning in Semantic Kernel is interesting, and very cool.

If you recall from Part 1, a Kernel app composes together Kernel Plugins and Kernel Functions to achieve a desired outcome.

But how do these Plugins and Functions work together to solve the problem?

Planning.

Semantic Kernel uses a concept called Planning to decide when, how and with what arguments to call a Kernel's plugins and function.

More accurately, Semantic Kernel uses AI itself, a LLM to draft a plan to achieve that goal. This plan is a series of plugin/function calls to get to the final result.

Old vs. New

Interestingly, at the time of this writing, Semantic Kernel recommends that you use a feature of LLMs called "function calling" to create a plan for a user using your AI agent.

The "old" way was to use built in types for "Stepwise" and "Handlebar" planners is superseded in favor of using LLMs that support "function calling".

Note, that "function calling" requires your Chat Completion service to use an LLM that supports it.

Planning Example

Note, this Polyglot Notebook uses the "old" way with a Handerbars planner.

The subsequent Notebook in the Semantic Kernel repository makes use of the SummarizePlugin and WriterPlugin which are parameterized prompt templates.

The SummarizePlugin has function prompts for making abstracts readable, note generation, summarization, and topics.

The WriterPlugin has prompt functions for several things such as email, acronyms, novels, stories and poems.

As before, a basic Kernel app is constructed with an OpenAI chat completion service.

The next steps, rationally, are to configure the the Summarize and Writer Plugins within the Semantic Kernel.

Lastly, you can use the planner object, passing in your configured Kernel along with your request to get a plan of execution.

For this example, you are asking your Kernel to genereate multiple date ideas communicated using a poem.

As you can see, there are two steps to the plan.

The final step is to execute the generated plan using the object returned by Semantic Kernel's planning step. This gives you the desired output.

As a little twist, the Notebook has you add an inline function to rewrite output in a Shakespearean style.

Which you can add to your Kernel app, and then write write the date idea peoms in the style of Shakespeare.

Analyzing the Example

So what's going on here?

Looking more closely at the defined prompt of the Rewrite function in the Writer plugin, you see inputs for $style and the actual $input of the user request.

The planner uses the context given by the user describing how it is Valentine's Day and that you what date ideas written as a poem. It then is able to use the LLM to determine which values to extract from the user's request context, and configure that as the input Kernel Arguments for the Rewrite function in the Writer plugin.

You're essentially using the LLM to figure how to better prompt the LLM. Which is very cool.

Memory

Moving on to the next Juputer Notebook in Semantic Kernel's GitHub repository, we get to Semantic Memory, which is a way to persist state within your Kernel application for more interesting applications.

The start of the notebook is the familiar builder pattern setup for a Kernel application.

Conceptually, a Kernel plugin for Memory is used, but also a Memory storage technology.

The Polyglot notebook uses an in-memory solution called VolatileMemoryStorage which is NOT persistent across sessions but there are different storage implementations that give you persistence.

Embeddings

At the time of this writing, the Microsoft documentation for embeddings is limited. Looks like they are in the process of adding it.

I'll update this section in the future.

Memory as a Data Structure

The starting point is to manually add some contrived examples of embdedding information about a person.

In short, generating emdeddings is all about turning your information into a set of floating point numbers that correspond to relevenace and meaning.

Here you see the about of turning those memories into vector embeddings.

It's important to note that an OpenAI model for generating embeddings was used. You can see it was configured during initialization of the memory object.

Total Recall

It's nice that the tutorial refers back to the chat bot notebook from Part 1 so you can see how memory builds on top of prior work.

Semantic Kernal has a native TextMemoryPlugin with a recall function that returns the most relevant memory currently found in the storage medium backing Semantic Memory.

Add that to your Kernel app by importing it as an object.

Now you can recreate the chat function that loops through your converstaion storing history, but using a prompt that is primed with the contrived information about yourself.

As always, be sure to set your Kernel Arguments to pass in.

Essentially, you now have a chat bot that has information about you, so you can ask it context senstive questions about yourself.

This demonstrates how you can use Semantic Memory, which is part of Semantic Kernel, as a way to inject context specific information.

Using this context like a user's personal information can provide a more personalized chat experience.

But this information is just contrived sample information, what is special about that?

Not much, but you can extend this concept to your own documents to get a better experience interacting with your data.

Talk to Me Goose

Have you ever wanted to chat with your data?

I have. And I think you will too.

Semantic Kernel provides a way to create embeddings of your own documents which you can then use within your Kernel app.

The notebook first has you build memories using files from the GitHub repository.

As before, you construct a memory builder object using the VolatileMemoryStore as the storage medium. You can see an example of that above.

Iterating over the array of URLs, you add them to your memory data structure.

Now you can search the memory construct for your documents and elicit information about it using natural languange.

In the notebooks example, you can get a list of Polyglot notebooks by saying how much you like them when asking how to get started.

It's cool to imagine create a chat bot or other AI agent that can engage with your own data.

Conclusion

Getting deeper into the Semantic Kernel SDK, planning, memory and embeddings provide an enriching element to the art of design an AI agent.

Looking ahead to future parts, it looks like there are additional integrations with other AI services such as OpenAI's DALLE image generation service.

For details on getting started, see Part 1 of this series.

You can read Part 2 - Planning, Memory & Embeddings here.

Intro

Semantic Kernel is an open source software development kit (SDK) from Microsoft for building Artificial Intelligence (AI) agents.

As of July 2024, programming languages supported are C#, Python and Java.

I prefer C# and am fascinated by the world of AI and these so-called AI agents. So Semantic Kernel is a natural choice for exploring this niche of AI.

Background

I have a rudimentary working knowledge about AI and Large Language Models (LLMs), but otherwise, I am currently a complete beginner.

I like solving problems and am excited about the kinds of problems that AI might help me to solve.

My review of Semantic Kernel is less an authoritative opinion about it's efficacy, and more of an exploration into whether I can use it to solve certain problems I encounter on a regular basis.

This review is me getting started with Semantic Kernel's step-by-step walk through.

Tools

I am using Visual Studio Code and Git to clone the open source repository: https://github.com/microsoft/semantic-kernel.

Step-by-Step

The GitHub repository has a folder with several Polyglot notebooks, which this is my exposure to and think they are quite nice.

If you've never used a Polyglot notebook before, it's a cross between a Wiki page and a REPL app. You can read along with written documentation and then execute blocks of code inline with the documentation.

Very nice.

Getting Started

The Semantic Kernel SDK comes with support for loading LLMs from Microsoft Azure and OpenAI. All you need to provide is a handful of configuration settings like and API key and LLM model name.

For my walk through I used an OpenAI key to get started quickly. Long term, I hope to use Semantic Kernel with a local LLM but that will have to wait.

Programmatically, the SDK the latest dotnet prescribed builder patter to construct an Semantic Kernal type application.

With just a few lines of code, you're up and running with a Semantic Kernel specific application that is configured to use an Azure or OpenAI ChatCompletion service.

Pause for Perspective

Semantic Kernel essentially is providing a so-called "Kernel" application that allows you as the developer to orchestrate interaction with various services, both AI in nature and your own home grown services.

There are a few terms you need to familiarize yourself with like Semantic Plugins and Semantic Functions, which I call Plugins and Functions moving forward.

The objective is to compose together the right plugins and functions to solve your specific problem.

Back to Tutorial

The first Kernel Plugin you use is the "Fun Plugin" which essentiallys asks for a family friendly joke.

Prompts, From File and Inline

A core idea with Semantic Kernel is to construct an AI chat "prompt", which is specific instruction text you give to a LLM to describe the output you want it provide you.

Semantic Kernel prompts can exist in individual files on disk, or constructed in code also known as "inline".

Let's Chat

Now for more interesting stuff, which is a chat like experience that you'd expect from something like ChatGPT.

The key idea here is that these Kernel prompts have a special syntax where you as the developer can specify parameters that your user can pass in as arguments to the Kernel.

Semantic Kernel describes these as "Kernel Arguments" which is probably a misuses of the term "argument" but that is not important here.

What's important is that you can inject dynamic information into your Kernal's prompt. For example, see the following here where you have two Kernal Arguments. A $history for injecting your chat history information, and a $userInput for the next user message for the chat bot.

The rest of this notebook, you construct your Kernel Arguments object which is essentially a key/value dictionary and pass it to your constructed chat bot prompt to get a book suggestion.

This is surprisingly very little code to achieve exceptional results that, with a little effort, could rival the user experience for these large LLM services.

The rest of the notebook tutorial demonstrates how you can write a singular function to generate a chat response, store it in the history argument value and keep entering user input to get context aware, converstational output.

Again, with very little code, this is a very cool experience.

Conclusion

This is only my first look at the Semantic Kernel SDK.

I have a lot more of their step-by-step Polyglot Notebooks to get through, but overall I'm happy what you can achieve with very little code running Semantic Kernel.

Check back for future parts where I continue working through how Semantic Kernel can solve everyday problems.

• Part 2 - Planning, Memory & Embeddings here.

This server uses Docker Compose to run a couple services I use locally.

Make Sure it's Plugged In

Logging into the server, I notice that restarting docker compose caused an error.

$ sudo docker-compose up -d
Error response from daemon: Conflict. The container name “/foo” is already in use by container “edjd98889d8dfddd9090998ddd09898ddd234234”. You have to remove (or rename) that container to be able to reuse that name....

The top posts from a Google search discussed checking to see if anything was using that port, which was probably not the underlying cause.

Troubleshooting Time

So next came a set of troubleshooting steps.

First up was to check the docker-compose.yaml file to see if anything was out of the ordinary.

$ vi docker-compose.yaml

Nothing unusual in the file, but I noticed that when I closed it, there was a quick flicker of vi's command status bar that flashed red showing some kind of error.

I re-opened the file, the did a :qw vi command to save the file and I received an "unable to save...out of disk space error".

Getting Somewhere

Alright, let's check the disk usage.

$ df -h
\Filesystem                                                   Size  Used Avail Use% Mounted on
tmpfs                                                        392M  1.5M  390M   1% /run
/dev/mapper/server--vg-root                          193G   0  193G  100% /
tmpfs                                                        2.0G  4.0K  2.0G   1% /dev/shm
tmpfs                                                        5.0M     0  5.0M   0% /run/lock
/dev/sda1                                                    470M  254M  192M  57% /boot
nas.local:/volume1/serverbackups/server   14T  3.0T   11T  22% /mnt/nfs/serverbackups/server
tmpfs                                                        392M  4.0K  392M   1% /run/user/1000
/home/user/.Private                                      193G   0  193G  100% /home/user

Yep, 100% of the file system was used up which was very unexpected.

After several attempts to delete files, I was still at 100% file system in use.

Using series of the following command:

$ sudo du -sch *

I concluded that the /var/lib/docker folder was using an unusual amount of disk space.

So it made sense to clean up docker.

Trimming the Fat

After years of use, I knew I had plenty of old images that could be removed.

So I ran the following command.

$ sudo docker system prune
...
deleted: sha256:715a1b962166ede06c7a0e87d068a4b686e6066e0eca5ecab6f4d6cfab2121fe
deleted: sha256:97ab3baee34d0c75ee10e65c63a06cbc87d20d695c17d14ad565d4ff1b8dc2ca
deleted: sha256:9f54eef412758095c8079ac465d494a2872e02e90bf1fb5f12a1641c0d1bb78b

Total reclaimed space: 15.67GB

The output of the command show that I reclaimed 15.67G of space.

Awesome, time to start everything back up.

$ sudo docker-compose up -d
Creating network "user_default" with the default driver
Creating service1  ... done
Creating service2 ... done
Creating service3 ... done

After everything restarted successfully, I could see that my disk file usage was back to normal, down almost 100G.

$ df -h
Filesystem                                                   Size  Used Avail Use% Mounted on
tmpfs                                                        392M  1.5M  390M   1% /run
/dev/mapper/server--vg-root                          193G   91G   92G  50% /
tmpfs                                                        2.0G  4.0K  2.0G   1% /dev/shm
tmpfs                                                        5.0M     0  5.0M   0% /run/lock
/dev/sda1                                                    470M  254M  192M  57% /boot
san.local:/volume1/serverbackups/server   14T  3.0T   11T  22% /mnt/nfs/serverbackups/server
tmpfs                                                        392M  4.0K  392M   1% /run/user/1000
/home/user/.Private                                      193G   91G   92G  50% /home/user

Conclusion

Hard to say what happen but given that this occurred after a power outage, it's likely that the servers in my homelab did not start back up in the correct order.

Specifically, I think my network attached storage (NAS) server was off when the broken server tried to mount a drive used by the docker services.

This likely caused some sort of run away crash loop that filled up logs files in all the docker volumes.

Then after everything started successfully, the docker services probably clean up the log files or any large mounted files it was hanging onto.

I didn't even notice.

I happened to check one of my websites and notices nothing came back.

Early notification when there is a problem in my homelab is a discussion for another time.

Today, I want to run through what I did to get my system back up and running.

Symptom

My website are down, getting 404 not found or 503 service unavailable.

Troubleshooting

I first log into my kubernetes master node and check to see if my pods are running.

Note: that k is an alias for kubectl.

$ k get all
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?
The connection to the server 127.0.0.1:16443 was refused - did you specify the right host or port?

Oh boy, now what?

I have a few years experience running a kubernetes cluster and am familiar with expired certificates and what a pain it is to fix them.

I happen to be running a lightweight distribution of kubernetes called microk8s. I found their documentation helpful regarding certificates.

$ sudo microk8s refresh-certs --help
[sudo] password for :
Usage: microk8s refresh-certs [OPTIONS] [CA_DIR]

  Replace the CA certificates with the ca.crt and ca.key found in CA_DIR.
  Omit the CA_DIR argument and use the '--cert' flag to auto-generate a new CA
  or any other certificate.

Options:
  -c, --check  Check the expiration time of the installed certificates
  -e, --cert   The certificate to be autogenerated, must be one of ['ca.crt', 'server.crt', 'front-proxy-client.crt']
  -u, --undo   Revert the last refresh performed
  -h, --help       Show this message and exit.

$ sudo microk8s refresh-certs -c
The CA certificate will expire in 3276 days.
The server certificate will expire in 364 days.
The front proxy client certificate will expire in -9 days.

Whoops, I can't believe my sites have been down for 9 days.

Here's the fix to renew the expired certs:

First the server.crt.

$ sudo microk8s refresh-certs -e server.crt
Taking a backup of the current certificates under /var/snap/microk8s/6673/certs-backup/
Creating new certificates
Signature ok
subject=C = GB, ST = Canonical, L = Canonical, O = Canonical, OU = Canonical, CN = 127.0.0.1
Getting CA Private Key
Restarting service kubelite.
Restarting service cluster-agent.

Then the front-proxy-client.crt.

$ sudo microk8s refresh-certs -e front-proxy-client.crt
Taking a backup of the current certificates under /var/snap/microk8s/6673/certs-backup/
Creating new certificates
Signature ok
subject=CN = front-proxy-client
Getting CA Private Key
Restarting service kubelite.

Now, I can recheck to make sure everything looks good.

$ sudo microk8s refresh-certs -c
The CA certificate will expire in 3276 days.
The server certificate will expire in 364 days.
The front proxy client certificate will expire in 364 days.

Awesome! Now let's see if I can access my kubernetes components.

$ k get nodes
NAME            STATUS   ROLES    AGE    VERSION
kbndev01   Ready    <none>   366d   v1.26.15
kbndev02   Ready    <none>   366d   v1.26.15
kbmdev01   Ready    <none>   373d   v1.26.15

Excellent! Now I try to access one of my sites but still nothing is coming up.

More Solutions, More Problems

After several different troubleshooting steps of restarting services and rebooting all the nodes, I find this.

$ k get nodes
NAME            STATUS     ROLES    AGE    VERSION
kbmdev01   Ready      <none>   373d   v1.26.15
kbndev01   NotReady   <none>   366d   v1.26.15
kbndev02   NotReady   <none>   366d   v1.26.15

Why are these nodes not ready?

Do I need to refresh the certs on them as well?

The answer is yes.

I log into each node individually and run the refresh-certs commands to fix the certs.

Then after a few seconds, all the nodes are ready.

$ k get nodes
NAME            STATUS   ROLES    AGE    VERSION
kbmdev01   Ready    <none>   375d   v1.26.15
kbndev02   Ready    <none>   368d   v1.26.15
kbndev01   Ready    <none>   368d   v1.26.15

After that, all my sites were once again accessible.

Conclusion

If your kubernetes certificates are expired, especially when using the microk8s distribution, be sure to renew all the certs on all the nodes.

You can use a private NuGet repository in your docker build process. The best, most deterministic way to do this is to add a NuGet.config file to your repository with details.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <clear />
        <add key="NuGet" value="https://api.nuget.org/v3/index.json" />
        <add key="myfeed" value="https://www.davidpuplava.com/nuget/index.json" />
    </packageSources>
    <disabledPackageSources />
</configuration>

Consider the following Dockerfile for building an ASP.NET web application. This work if your NuGet repository does not require authentication.

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
LABEL stage=build-env

WORKDIR /app

# Copy and build
COPY ./src /app
COPY ./NuGet.config /app
RUN dotnet publish /app/MyApp.Web -c Release -o ./build/release --framework net8.0

# Build runtime image
FROM mcr.microsoft.com/dotnet/aspnet:8.0
EXPOSE 80
ENV ASPNETCORE_URLS http://+:80
WORKDIR /app
COPY --from=build-env /app/build/release .
ENTRYPOINT ["dotnet", "MyApp.Web.dll"]

Accessing Securely

If your private NuGet feed requires authentication, you have a few options for how docker build can authenticate with your feed. One options is to store the credentials in plain text in the NuGet.config file but this is not very secure. Plus your intermediate layers from your docker build process will store this information.

You can environment variables to pass this information into your docker build process without storing secrets in your NuGet.config file.

You can take advantage of NuGet's default environment variable support NuGetPackageSourceCredentials_<package-name> where <package-name> is the name of your private repository. When this environment variable is set, NuGet will automatically use it for authentication to that repository.

The formation for the value of the environment variable is Username=...;Password=...;. You can also add ValidAuthenticationTypes=Basic if you desire. You'd do this if you want to explicitly control how NuGet should authenticate against your repository.

Consider the following example.

Username=myfeed-pat;Password=SuperSecretPWD!;ValidAuthenticationTypes=Basic

To configure, open Window's System Environment variables and click "New...". Enter the special environment variable name, and value.

In our working example we have a private NuGet repository named myfeed that is accessed with a personal access token username/password of myfeed-pat/SuperSecretPWD!.

Use Environment Variables in Docker Build

After you configure your environment variable, you can use it in your Dockerfile by adding these two lines. The first line adds an argument to your docker build file. The second line assigns that argument value to your docker build's environment variable context.

ARG NuGetPackageSourceCredentials_myfeed

ENV NuGetPackageSourceCredentials_myfeed=$NuGetPackageSourceCredentials_myfeed

Here is the final Dockerfile using envinroment variables to authenticate against a secured private NuGet repository.

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
LABEL stage=build-env

ARG NuGetPackageSourceCredentials_myfeed

WORKDIR /app

# Copy and build
COPY ./src /app
COPY ./NuGet.config /app
ENV NuGetPackageSourceCredentials_myfeed=$NuGetPackageSourceCredentials_myfeed
RUN dotnet publish /app/MyApp.Web -c Release -o ./build/release --framework net8.0

# Build runtime image
FROM mcr.microsoft.com/dotnet/aspnet:8.0
EXPOSE 80
ENV ASPNETCORE_URLS http://+:80
WORKDIR /app
COPY --from=build-env /app/build/release .
ENTRYPOINT ["dotnet", "MyApp.Web.dll"]

Passing Credentials

You can now pass your build server's environment variable into docker build to set the docker build processes environment variable during the build process with --build-arg.

Here is the docker build command.

docker build -f .\Dockerfile -t davidpuplava.com/nuget --build-arg NuGetPackageSourceCredentials_myfeed=$($Env:NuGetPackageSourceCredentials_myfeed) .

Summary

You can securely access a private NuGet repository with docker build using environment variables to avoid storing secrets in source control.

cat (Get-PSReadLineOption).HistorySavePath | findstr cd

You will see result similar to the following.

cd .\test\
cd .\Test.Mvc\
cd C:\git\Test
cd C:\git
cd .\Test\
cd ..
cd .\test-old-hyperadmin-etc\

Story of my life.

The Symptom

Changes I'd make and run locally worked as expected on my dev machine. But when I'd deploy them to production, they wouldn't be there.

More Context

More specifically, I was forked an open source library to make a change I needed.

I then rebuilt the Nuget package, stored it in a local nuget repository source and referenced that local nuget package in my app.

I would run the app locally on my dev machine and, boom, the change works flawlessly.

To move the change to production, I built a new docker image and deployed that new image to the production server.

AHHHH!!! The change wouldn't show up, even though I know I'm running the new image.

The Problem

After many hours of insanity, quite literally trying the same thing over and over expecting different results, I finally realized the cause of my problem.

The Dockerfile I used to build the docker container has no knowledge or access to the local nuget feed on my dev machine.

Why did it take so long to realize this?

Because the opensource library I was using is available publicly on nuget.org. And the source of my frustration was that I did NOT, I repeat, I DID NOT, use a different/unique/not-currently-used-on-nuget-dot-org version number (laziness, I guess).

So, everything build without error. But the build process would grab the nuget package from nuget.org which did NOT have my changes that I packaged into the local nuget feed version of the same library.

The Solution

I solved this problem by

1. Copying the local nuget feed packages to a folder in the root of my git repository.
2. Adding a COPY instruction to the Dockerfile to ensure that the local nuget packages are availble during the build step of the docker build.
3. Adding a RUN nuget add source command to add the local nuget feed as a nuget package source during the docker build.

After I made that change, the new docker image work as expected in the production environment.

Here is a better command I found for removing only those bin and obj folders.

Get-Childitem -Path . -Include bin,obj -Recurse -Directory | Where-Object { $_.FullName -notmatch '\\(node_modules|packages)\\' } | Remove-Item -Recurse -Force

Here is breakdown of the command.

Get-ChildItem -Path . is used to get the file and directory objects in the current path. Optionally, you can change the argument passed to -Path to which ever path you like. If your projects folders are solely in the ./src directory you can use Get-ChildItem -Path ./src instead to narrow your directory traversal.

-Include bin,obj filters the child items to just the bin and obj folders. If you want to include different directories you can change the argument passed to -Include.

-Recurse -Directory recursively goes through down through all the directories.

Note that this above command is three separate commands piped together. The first command Get-Childitem -Path . -Include bin,obj -Recurse -Directory is then piped to Where-Object { $_.FullName -notmatch '\\(node_modules|packages)\\' }.

Here is a breakdown of the Where-Object { $_.FullName -notmatch '\\(node_modules|packages)\\' } command.

Where-Object { ... } is an additional filter on all the directories that the first command is recursively navigating through. Everything inside the curly braces { } is how it filters.

$_.FullName -notmatch '...' is doing a regular expression operation where -notmatch means that any directory paths that do NOT match the filter string (the thing between the single quotes) will be include. At a high level, this whole command is how you can exclude certain directories from being considered when deleting the bin and obj folders. In particular, the node_modules directory is ecluded because that is related to gulp operations and the NPM packages will download packages that have bin and obj folders. This is turn disrupts the gulp tooling which is not desired. Additionally, the packages folder is also excluded because that is sometimes used by nuget to store downloaded packages. I skip it here for the same reasons as I do for node_modules.

Essentially the entire second command Where-Object { $_.FullName -notmatch '\\(node_modules|packages)\\' } is about skipping those directories that have a bin and obj folder I want to keep before sending to the 3rd and final command Remove-Item -Recurse -Force.

The final command Remove-Item -Recurse -Force is what does the deleting of the directory.

The Remove-Item is the PowerShell command. The -Recurse will recursively delete the child items like files and additional folders. The -Force option deletes the directory without requiring you to confirm you want to delete a non-empty directory.

That's it. That's the command that I use to recursively remove bin and obj folders. Use this rather than the implementation below, which is there for reference.

Original Implementation (DO NOT USE, kept here for legacy reasons)

Run these commands to remove all the bin and obj folders.

get-childitem $path -include bin -recurse | remove-item
get-childitem $path -include obj -recurse | remove-item

Special thanks to this post for giving me the answer

Why?

Sometimes in the .NET solution, even the batch build's clean and rebuild commands do not clear out all files from the bin and obj folders. Reason being is the msbuild will not remove files it is not aware.

The exact scenario I need to solve was that I had changed the version number of a nuget library but when I rebuilt the solution, the old assembly file was in the bin folder and that was used rather that you new downgraded version.

More precisely in my case, I had repackages a new version of a local nuget package using the same exact version number. Because...reasons.

Conclusion

When it doubt, clean it out.

I've waited a long time for AT&T Fiber to arrive in my neighborhood and it is finally here. Naturally I signed up as soon as I could and scheduled my fiber installation at their next available slot.

The installation went great. The AT&T technician, Richard, was very knowledgeable and was able to install the AT&T Gateway in my server closet on the other side of my house from where the fiber entered my home.

AT&T's default configuration is to leave you with working WIFI and you can connect to and start browsing the internet. This makes sense because most people do not wire their houses with ethernet.

As a software developer tech nerd, I'm not most people. My appetite for punishment lead me to wire my house for ethernet, at least for those devices that benefit from high throughput like my work computer and streaming devices.

Naturally, I have my own router with a custom configuration to handle all of that.

Additionally, this router needs to "live on the edge" as they say to handle some services that I host out of my own homelab.

What this means is I need to "bring my own router" to use with the AT&T Gateway that was installed.

Plug it in, and it works! Sort of...

To use your own router with AT&T's BGW320-505 gateway, you simply plug it into one of the yellow 1Gb ethernet ports on the back of the device. The blue 5Gb strongly recommends that you use CAT 7 ethernet, which I honestly thought was a made up category of ethernet but turns out I'm just too old to keep up with technology.

You then use a computer to connect to your AT&T gateway accessing the default address http://192.168.1.254 to see your Gateway settings.

Navigate to the Firewall page, and then the IP Passthrough subpage and do the following:

1. Set Allocation Mode to Passthrough to allow the AT&T Gateway to pass your WAN IP address to your own router.
2. Set Passthrough Mode to DHCPS-fixed so you can specify your router as the client to receive the passed through WAN IP (rather than the first client that connects).
3. In the Passthrough Fixed MAC Address enter your router's MAC address which you can find on the back or side sticker physically attached to your router. You can also click the Choose from list to see the list of connected devices; if it's obvious which device is your router, select it to automatically fill in the Manual Entry box with your router's MAC address.

After that, I restarted both the AT&T Gateway, and my own router and everything worked great!

Until...

I Couldn't Connect to the Internet

When I woke up the next morning, nothing worked. I couldn't connect to the internet, I couldn't connect to the AT&T Gateway, and I couldn't connect to my own router.

I did the usual steps of checking my ipconfig settings, ping and tracert but nothing seemed to work.

Oddly enough, email and a couple other apps would update with notifications as if it sort of worked, but everything was intermittent.

To connect to my AT&T Gateway, I had to explicitly connect to the default WIFI and then everything from that computer worked; I could browse the internet, reach the router etc.

I finally found AT&T's article on how to use your own router and that helped me fix the issue.

The key issue for me was that AT&T's Gateway used the same subnet 192.168.1.0/24 as my own router's subnet.

Even though the ranges didn't overlap, the power struggle was real and cause all sorts of issues.

The key point from AT&T's article was this:

After I read that, resolving the issue was as simple as changing the default subnet for the AT&T Gateway.

Fixing the Problem

Follow these steps to resolve the subnet conflict.

1. Navigate to your AT&T configuration page at http://192.168.1.254.
2. Go to the Home Network tab.
3. Select the Subnets & DHCP sub page.
4. Change the Device IPv4 Address to use a different IP address, for example 192.168.3.254.
5. Update the DHCPv4 Start Address to use the subnet, in this case 192.168.3.x, so enter the value 192.168.3.64. Notice here, I'm only changing the third octet.
6. Do the same for the DHCPv4 End Address, here I enter 192.168.3.253.
7. Click Save at the bottom of the screen.

I restarted both my AT&T Gateway and my router, just for good measure but I don't actually think it's necessary.

After I made the above change, everything started working.

One Last Thing

Since I already had a WIFI Access Point, I wanted to avoid having the AT&T Gateway Access Point WIFI radio's from interfering, so I went ahead and turned them off.

1. Navigate over to the Home Network tab.
2. Select the Wi-Fi sub page.
3. Click the Advanced Options click to see more options displayed.

4. From here, I set the Wi-Fi Operation to Off for the 2.4 GHz Wi-Fi Configuration.

5. Next, under the 5 GHz Wi-Fi Configuration section, I set Wi-Fi Operation to Off.

6. Lastly, don't forget to click the Save... button.

7. When you see this warning screen, go ahead and click the Continu. button to disable the Wi-Fi radios on the AT&T Gateway.

That's a Wrap!

Thanks for reading and I hope you found this article useful if you run into the same issues connecting your own router to an AT&T Gateway.

Special thanks to this YouTube video as well which confirmed what I thought need to be done to use my own router.

If you receive a file exceeds the maximum upload size in Orchard Core and you're certain the settings allow for the size image you've uploaded, check to see if you have a reverse proxy the is erroring out too big header response sent from the upstream server.

In my case, the problem was Nginx and not OrchardCore.

502 Bad Gateway from OrchardCore running in Kubernetes

In particular, I run Orchard Core in a Kubernetes cluster configured with an Nginx Ingress Controller.

My symptom was that I couldn't upload images larger than 768 KB even though my OrchardCore settings allowed images up to 4 MB.

After solving a different 502 Bad Gateway error when logging into Orchard, I learned that the Nginx Ingress Controller was erroring out because it was running out of buffer space.

From that troubleshooting experience, I learned that I can configure individual Ingresses to increase that buffer size.

Increase Nginx Buffer Size with Annotations

The solution was to add these annotations to my ingress for my particular OrchardCore site.

    nginx.ingress.kubernetes.io/proxy-body-size: 30m
    nginx.ingress.kubernetes.io/proxy-buffer-size: 256k
    nginx.ingress.kubernetes.io/proxy-buffering: 'on'
    nginx.ingress.kubernetes.io/proxy-buffers-number: '4'
    nginx.ingress.kubernetes.io/proxy-max-temp-file-size: 1024m

And that's it!

After I deploy the above ingress annotations to increase the Nginx Ingress controller buffer size, I could sucessfully upload image files up to the size that was configured in OrchardCore media settings.

Annotate your Nginx ingress controller to increase the proxy buffering size for your upstream server. By default they're inadequately small and you end up with the following errors in your reverse proxy logs.

For the following symptoms

• You access your web application, service or API and get a 502 Bad Request even though you're certain the operation should work as expected.
• You also enable verbose logging and you're not seeing any log message or errors that would indicate what the problem might be.

The more involved answer follows.

Getting a 502 Bad Request for OrchardCore Running in Kubernetes

Talk about deep troubleshooting. This one was several days in the making.

I was setting up a new website in my OrchardCore deployment. The tenant loaded just fine but I was getting a 502 Bad Request when trying to log in.

This was strange because the authentication code is all native Orchard Core (and ASP.NET) default stuff. But I had recently upgraded to OrchardCore 1.8.0 so maybe there was a problem with that.

I enabled verbose application logging and not event a mention of any kind of problem or error.

WTF.

Check Kubernetes Nginx Ingress Controller Logs

I run OrchardCore in a Kubernetes cluster.

This cluster uses an Nginx Ingress controller to handle incoming requests for this particular client application.

After seeing nothing interesting in OrchardCore's applicaiton logs, and also nothing in the Kestrel host logs, I decided to check my reverse proxy or ingress controller logs.

For my MicroK8s setup, my ingress pods are in their own ingress namespace. There are multiple pods and I want to see the logs for all of them. Luckly, they share a common label nginx-ingress-microk8s.

Identify the Upstream Sent Too Big Header Error

Run the following (or similar) command to see your ingress logs.

kubectl logs -l name=nginx-ingress-microk8s -n ingress 

And search for upstream sent too big header like in the following log output.

2024/01/10 15:30:05 [error] 711#711: *4185444 upstream sent too big header while reading response header from upstream, client: 10.1.40.193, server: www.*********.com, request: "POST /Login HTTP/2.0", upstream: "http://10.1.x.x:80/Login", host: "www.*********.com"

Boom, found it. After a little web searching you find out that Nginx has very small proxy buffers by default and they need to be increased.

Add Ingress Annotations Configuration to Increase Nginx Proxy Buffer Size

The solution was to add these annotations to the ingress configuration.

    nginx.ingress.kubernetes.io/proxy-body-size: 30m
    nginx.ingress.kubernetes.io/proxy-buffer-size: 256k
    nginx.ingress.kubernetes.io/proxy-buffering: 'on'
    nginx.ingress.kubernetes.io/proxy-buffers-number: '4'
    nginx.ingress.kubernetes.io/proxy-max-temp-file-size: 1024m

And that's it. After I added and deployed these annotations to the Nginx Ingress for my application, I was able to successfully log in no problem.

Incidentally, this also fixed my OrchardCore "file exceeds the maximum upload size" error, which you can read about here.

Why?

The Ender-3 Pro is very easy to put together. It prints very nicely, and really gives a beginner a sense of all the things that go into 3D printing.

It also gradually introduces you to some of the more challenging sides of 3D printing, so you can troubleshoot, them one at a time. For example, any of the axis' might start acting weird, and you'll have to work through solving that.

Also, the manufacturer, Creality, is big on their stuff being open source which is nice.

It's also priced just shy of $300 USD which is more affordable than some of the other "beginner" printers out there.

The price per quality ratio is the biggest factor. I strongly believe the Ender-3 Pro gives you the best value for your money.

Some Caveats

I have only owned two different 3D printers, so my sample size if very small.

That said, I knew nothing about 3D printing before I received the Ender-3 Pro as a gift several years ago.

And I was up and running and printed my first thing a couple days later.

I really think this is the best 3D printer for a beginner or hobbyist.

In this post, I cover some of my favorite features of OrchardCore 1.7.

Two-Factor Authentication

OrchardCore now supports 2-factor authentication (2FA) with email and SMS messages. But by far my favorite feature is support for authenticator apps.

You can now use your prefered authenticator app such as Microsoft Authenticator or Google Authenticator and use one-time passwords (OTP).

You can enable two-factor authentication by navigating to the Features section of the Admin area of your OrchardCore site. Search for two-factor to filter the feature list. There you can choose to enable any and all methods: Email, SMS and Authenticator App.

Once enabled, you see a new menu item named Security under the user down carret menu in the upper right hand corner.

On the TwoFactor configuration setting screen, you see a list of the two-factor methods you enabled from the features screen. If you enabled Authenticator app, you'll see that option. If you have yet to set it up, you can click the Add button to get started. If already setup, you'll see Set and Reset buttons.

Click Add to get to the Configure Authenticator App screen.

On the configuration screen, you'll see everything you need to get setup. If you already have an authenticator app, you scan the QR code or type in the key code to your app. If you don't have an app, you'll see links to the Android and iOS app stores to get one set up.

Once your authenticator app has an entry for your site, you can look at the current OTP code and key it in step 3 on the screen to verify that the authenticator app was registered correctly.

After verifcation, you are all set up with 2FA using an authenticator app. When you log in with your password you'll be prompted for the one time password.

And that's it!

The best part, as is with all features in OrchardCore, is that this works for all your tenants (if enabled). It just works. And it's great.