Vibe coding: supply chain insights

My day job requires me to deal more with non-technical stuff and it is pretty interesting to see the change that is currently happening with the introduction of AI. So I decided to play around with some vibe coding and well the desire to also take vertex synapse for a spin.

I was mostly curious about how easy it could be to maintain an API for insights into the supply chain of a government. This is by no means complete, but it allowed me the chance to further experiment with vibe coding as well as the hypergraph solution from Vertex.

This blog is mainly about how easy it is nowadays to go from idea to proof-of-concept.

You might be wondering why I chose the government, well mostly cause they are IMHO more transparent with their data and insights. This is also one of those ideas that could be managed and maintained by the national entity for cybersecurity. Would be pretty cool if all kind of entities in a country would be incentivised to provide data and receive the opportunity to query their supply chain risk. This could also be used by threat actors, but they can do that now anyhow.

The idea is pretty simple, can we create a graph of how companies interact with each other in terms of being subsidiaries, vendors etc. Can we then add software to it, to understand which software is being shared amongst companies? The more detail that we add to this graph the more accurate you’d be able to determine the impact of a supply chain attack on an organisational level.

Vibe coding made it very easy to create a proof-of-concept, mind you this is not production worthy. Then again that wasn’t the goal since I mainly wanted to validate if the idea would provide insights.

You can find the code for this experiment here. I’ve also attempted to create an AI friendly start here if you want to make it yourself fully from scratch.

The overall web application looks like this:

The web application offers exploring the Vertex Data Model as well as manually entering organisational data and then analysing that data. The loading of the data happens with external scripts. For now I’ve only inserted a limited data set:

That’s the nice thing about government transparency, you can investigate stuff and well help them out if you so desire. Based on that overview of websites you can then:

  • Resolve all the domains to IPs
  • Understand the relationship between the different government entities
  • Analyse the websites for software used

This does not fully cover the supply chain risk, if you want to improve the data set you can use the new government code environment:

Go through all repositories link them to the right government entities and update the software used. Bonus, if they have generated SBOMs you can improve the accuracy of your data set and thus your analysis.

Since the Dutch NCSC is pretty awesome, I’m gonna use them as an example in this vibe coded application and some analysis snippets. Let’s start the organisation view:

This image provides us with a small overview of domains, parent organisation etc.

Sometime a table is just clearer, even though I am a big fan of graphs. We can also quickly view the software used by the NCSC, how many other organisations also use it?

Nothing scary, but does provide interesting insights, we can zoom in on the specific URLs and organisations that for example use react:

The graph based exploration is of course a must have :p

Lastly, let’s say we want to hack some CMS system and understand what our impact would be, this would be an interesting view to look further into:

Observations on AI generated Remote DuckDB via HTTP with mTLS

The last post on some notes to myself, made me curious on how easy/difficult it would be to generate a very simple solution to make DuckDB accessible from remote locations. There are probably far better projects out there to remotely access your DuckDB files, but I mainly wanted to have an mTLS first solution, with no options to expose it without client certificates. In addition, I also wanted to keep it simple. You can find the code over here and I am trying/tried to keep the commit history informative in terms of what AI has been improving.

My main curiousity driven questions were:

  • Does it reduce time?
  • How ‘secure’ is the first version of generated code?
  • Can AI aid in iteratively improve the code?

Fair warning; I’m no AI nor LLM expert, I’m just experimenting as I go. For those wondering I’m using vscode with the github copilot plugin set on auto selection for models. I keep gemini chat in parallel to ask some questions and contrast the answers/solutions.

But, but but…your setup is missing all the context and/or agent files! Yes, I like re-inventing the wheel / making mistakes myself to better understand why other advise in a certain direction.

Does it reduce time?

At first sight it seems like it does, cause boy does it type fast! However, due to the lacking context / agent files (I only used simple prompting) you do have to tidy up / fix a lot after the first generated code. I guess it depends on how you measure time, but one way or another you need to invest time to get the solution you want in the way that you want it.

How ‘secure’ is the first version of the generated code?

I have to admit i didn’t include a ‘make sure it is secure’ part in my simplistic prompting. So I guess I should not be surprised the code isn’t secure by default. Reminds me of when I started programming during which I made (I still do) a lot of security mistakes. For example it generated certificates with sequential serial numbers, created files and directories with insecure permissions, didn’t lock down the DuckDB environment, etc. In a way it does mimic programmers who are not fully familiar with security intricancies of not only the language in which they are developing, but also the rest of the stack used for the solution being build.

Can AI aid to iteratively improve the code?

This it seems to be able to do, however don’t try to big of a leap in a single shot. For example, after the first version of the single file solution worked I wondered if I could single-shot it into a more maintainable version. I decided to have it plan & execute on creating tests and making it modular by splitting up the code. It got stuck on several tests in endless loops due to not overseeing requirements that had to be implemented first.

In smaller chunks it does a pretty good job of improving code and making suggestions. So I decided to improve the code bit by bit and trying to mirror this in the commit history.

Commit history of AI improvements after initial single-shot code generation

Idea behind this being to easily go back in time and see the type of improvements that AI suggests and makes when specifically prompted to review and improve code.

I have yet to split up the code, but then again, maybe this project is good enough like this for the learning purpose I setout to achieve.