Don't think you are, know you are.
15/03/2026
Dissecting the world of ai prompts and trying to figure out why AI thinks it's Neo from the Matrix.
When I first started using ChatGPT I saw a lot of people recommending to give your AI skills as prompts. "You are a senior developer, with blah blah blah..."
or
"You are an expert at marketing and in the top 1% of best Facebook marketing and ad campaigns, create me an ad that does..."
It seriously made zero sense to me, this is a large language model trained on trillions of bits of data, so why the heck would giving it a stupid suggestion like that actually improve the output?
Recently I've been noticing hallucinations. Not personally, but in Codex/Copilot. I think it was Copilot specifically I created an automatic trading bot on Binance exchange, it kept arguing with me that Binance API was broken and that I had to contact their support and lodge a ticket. I kept saying, that's unlikely because Binance was one of the largest exchanges in the world, and if their API was broken that would cause a lot of law suits.
Anyway, I had gone over the code and couldn't find anything wrong with it. So I gave up and contacted support. Sure enough it was a little path issue. I think from memory we had /[0] in the path which wasn't necessary.
That was my first introduction to 'Hallucinating AI'. Which seems to happen quite frequently. Knowing better now, I treat AI like a Junior Developer on crack. It is amazing good at pumping out code... It's just not that great. Ai is a coder, not a engineer. So if you're going to create anything with AI using code, expect to know what the fuck you're doing or you will have issues.
I know a couple years from now that won't be the case. Ai will get better and better. It's already going parabolic on the scale.
Source - ChatGPT generated bullshit.
Ok... maybe it's not going parabolic just yet. But it will...
The problems, endless problems
So I've been having the same issues for the past month or so with testing backtests with walkthrough backtesting trading strategies. First I made a simple trading bot, worked well, looked good. But I was giving it strategies and not getting the same results as others on Trading view. Same strategies, same coins, same time frame, same candle length, completely different results. Codex informed me it was due to some 1's and 0's. Also the way the UI was setout it wasn't great for backtests. This auto-trader had tickers everywhere, watching 20+ coins for potential trades. it wasn't setup for walkforward testing. So rather than paying TradingView $1200.00AUD for the ability to do walkforward testing I created another application to do just that. (yes both trading views lower plans, essential and pro don't have walkforward testing)
What is real? How do you define 'real'? – Morpheus
Now the issue I've been having is the same with the original application. The backtests just don't line up, so solution? Rather than telling AI it's a genius, which it isn't, I gave it specific 'skills'.
People have been selling these 'skills' all over the internet, every second ad on my Facebook video feed is trying to sell me an agency in a box. Assistant, Coder, Designer, Social Media Marketer, Email Marketer, etc...
But on GitHub we have the free - Agency-Agents, MarketingSkills, Claude's Superpowers... They are little more than wrapped up hubs of data to slightly improve the AI... some like Agency-Agents are just piles of garbage prompts wrapped up as an 'application'. yes, scripts of prompts. When I realised how fucking dumb this was I wanted to punch my monitor.
I don't believe that script prompting, or detailed prompting helps 1 iota. Just a sense I've had since the beginning. So what DOES WORK ? AGENTS.md, SKILL.md - these are files that give the AI extra context, extra content, and sometimes links directly to specific information that will actually help the AI.
It's still not much better than the AI googling for itself, but it is... slightly. So it's still worth doing.
What I did and what worked
Don't think you are, know you are. – Morpheus
AGENTS.md is a file you can place in every repo, it explains the context of the project, what you expect of the AI and links to SKILL.md or multiple SKILL.md files.
read about agents.md more here
SKILL.md is a file you explain the skill and link to more context. Generally a giant database of knowledge. You can use simple URL's to website documents as skills but it's far better to scrape the data and store it in a JSON file. So in the process of doing this I found a very handy little bit of script.
(yes 3 months into vibe coding, would have been nice to have at the start)
skill seekers automatic skill creation
Skill seekers is a free GitHub repo that allows you to build skills. If you have a large amount of data to scrape you can simply create a skill:
skill-seekers scrape https://docs.kraken.com/api/ -o ./kraken-developer --rate-limit 2.0
This then prompts you to give it a name, description, codeblocks, etc so it can create an aggregate file of the scaped data. So rather than the AI having to search it can simply grep the document. Which saves time and makes the AI feel smarter. But it's simply a time saver. It's like saying - "OK, before you do anything, search the internet". which it will do, but if you don't prompt it to every time it will stop doing that. At least with AGENTS.md and SKILL.md it works off that for every prompt because it's easy.
So now that I have 2 SKILL.md's built, Kraken-developer (the exchange API), and TradingView-developer (Pine reference v6), also some additional examples. Pine script is created by and used exclusively in Trading View, so it's silly to assume AI knows anything about it... It does however, but we shouldn't ever make assumptions about people or AI.
AGENTS.md ← behaviour rules for the agent
SKILL.md ← tells the agent when to use the skill
references/*.md/*.JSON ← actual technical knowledge
AGENTS.md
└──kraken-developer/
├── SKILL.md
└── references/
└── api.md
To be continued...
I'm going to do some tests with large prompts and generic "build me this" and output the results here. So come back soon. I expect it will make zero difference at least with OpenAI. Until I can afford the $300 to sub to Anthropic, that will have to do. 'Scientific as fuck', I know...