GPT Integrations

[Conqueror]

This thread is for discussion and updates related to GPT integrations in Mimir.

For those unaware you can now mention Mimir in Discord and ask it a question and it’ll respond in a fashion similar to the popular “ChatGPT”. Something of note is that the model is trained on data that ends in 2021 so it (despite telling you otherwise quite persistently in some cases) is unaware of anything that has happened since that time. So if you’re asking it general questions like “what gapped up?” or “What is going on with the debt ceiling discussions?” any answer you get will be as though the current time is 2021 (around March or so in most instances).

So again I say FOR FUCKS SAKE THE DATA ENDS IN 2021.

Now that we got that out of the way, we luckily can change this by slipping data to it in the requests. There are currently three types of requests built out:

Market Calendar Data

If you’re asking about the market calendar, you simply have to reference mcal in your question like so:

@Mimir using the mcal data for today, what could’ve caused a drop in the stock market?

This function currently supports referencing today and tomorrow in the query to switch between those specific sets of data. Week data will be implemented soon.

Earnings Calendar Data

Earnings calendar data can be referenced by using ecal.

@Mimir using the ecal data for today and your historical earnings data, what stock is most likely to have the largest move post earnings?

Similarly you can using today, tomorrow and week to switch between time ranges. (this changes the data that Mimir can see)

Candle Data

You can reference this by using candle data in your query and a stock ticker using the cashtag format of $ TICKER (no space same for the example below, just avoiding generating a watchlist entry as I haven’t built a way to avoid that yet.).

@Mimir using the candle data for $ SPY and comparing it to your historical data, would you say that the stock is oversold?

Due to the limited capacity of GPT the candle data is set to 30 hour candles. This limitation is unfortunate but it’s still useful for near term predictions and analysis. I’ve got some ideas on how to increase the capacity but this will be good for now.

[Navi]

So it can only hold 30 data points for calculations? (Well really for a candle I’m guessing there’s really only 4 major points per candle) so I guess more like 120 or so…

[Conqueror]

It can hold 4000 tokens total between the request and response. I’m still playing with it and I think I can fit more data in with some tweaking of how its formatted but here’s the description of tokens

What are tokens?

Tokens can be thought of as pieces of words. Before the API processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end - tokens can include trailing spaces and even sub-words. Here are some helpful rules of thumb for understanding tokens in terms of lengths:

• 1 token ~= 4 chars in English
• 1 token ~= ¾ words
• 100 tokens ~= 75 words

Or

• 1-2 sentence ~= 30 tokens
• 1 paragraph ~= 100 tokens
• 1,500 words ~= 2048 tokens

To get additional context on how tokens stack up, consider this:

• Wayne Gretzky’s quote “You miss 100% of the shots you don’t take” contains 11 tokens.
• OpenAI’s charter contains 476 tokens.
• The transcript of the US Declaration of Independence contains 1,695 tokens.

[Conqueror]

Noticed a bug in the market calendar function that is sporadically returning incorrectly calculated beats / misses. Going to be looking into it. One thing that we need to do is develop a hierarchy for the previous, consensus and forecast numbers that it’s processing so it can better interpret results when a data point has all three of those figures included.

[Conqueror]

Just a brief update. Progress on the chatbot related integrations has been on hold while I work on sentiment analysis and data scraping (first use being blind).

I did want to highlight three interactions with the bot last night during some research into Adani Group that highlight the benefits of it.

Q: An Indian company called Adani who is listed on NSE India as symbol ADE is crashing. What EU listed stocks or ETFs would likely be negatively affected by this crash? Think step by step and explain your answer step by step.
A: The EU listed stocks or ETFs that could be negatively affected by the crash of Adani Group on NSE India (symbol ADE) would be those that have a direct or indirect exposure to the Indian market. This could include ETFs such as the iShares MSCI India ETF (INDA) or the WisdomTree India Earnings ETF (EPI). Additionally, any EU listed stocks that have a direct or indirect exposure to the Indian market could also be affected. For example, companies such as Tata Motors (TTM) or Infosys (INFY) could be affected.

Q: Compile a list of banks that have given loans to Adani Group. Then, for each bank list any connections to the US that you can find. Provide specific examples.
A: According to reports, the following banks have given loans to Adani Group: State Bank of India, Bank of Baroda, ICICI Bank, Axis Bank, HDFC Bank, and Yes Bank. There are no known direct connections between these banks and the US. However, State Bank of India, Bank of Baroda, ICICI Bank, Axis Bank, HDFC Bank, and Yes Bank all have US subsidiaries that are members of the Federal Deposit Insurance Corporation (FDIC).

Q: If you had to pick one stock to short with relationship to Adani group what would you pick
A: Based on my analysis of the Adani Group and its connections to the stock market, I would recommend shorting the stock of ICICI Bank USA, a US subsidiary of the Adani Group. ICICI Bank USA is connected to American Express through its partnership with the bank, as well as through its membership in the Adani Group’s Global Payments Network. This makes it a good candidate for shorting, as it is likely to be affected by any changes in the Adani Group’s business.

While the data is outdated, the answers were extremely useful and pertinent.

The next step in chatbot dev is giving it the ability to conduct and digest Google searches when more information is needed to properly answer a question. Will provide an update when it’s reaady for testing.

[rd1]

Not sure if this is the right spot for this. We should create an alert for “ChatGPT” / Open AI integration in news related to tickers. I feel this may be the “XYZ announces AMZN partnership” causing a bump in stock prices. We saw that with BZFD and anecdotally I am looking at all companies scrambling to integrate this.

[Conqueror]

This is a good test case of this sort of thing.

I just deployed a test for this functionality. It’s looking for “OpenAI” or “ChatGPT” in the news headlines that Mimir has and if they exist in the headline, it’s forwarded to GPT to determine whether or not the headline is discussing a new investment, partnership or integration.

This is actually the first function that I’ve integrated that isn’t using the Davinci model (their most expensive and capable). This function is relying on Curie which is tier lower and 1/10th the price however at pretty solid cost to it’s capability.

Initial tests had pretty good accuracy so we’ll see how it plays out.

[rd1]

One of the options to consider is using GPT-J-6B: 6B JAX-Based Transformer – Aran Komatsuzaki
This is the open source version similar to GPT-3 model (curie) in terms of performance and parameter size.
GPT J 6B Demo - a Hugging Face Space by Narrativaai gives a faster demo than the original link to try out some tests

[Conqueror]

Ran a couple quick tests and it seems to be more in line with what I’ve seen from curie or babbage from OpenAI. They’re okay at super basic tasks but break down pretty quickly past that. Will continue to do some testing later this evening though, thank you!

I’ve begun to upgrade my home server to support running some of these models actually for testing’s sake.

[Conqueror]

Was talking a deeper look at this. Running it on hugging-face looks to be about $430 a month or so.

Currently looking at running it locally but I see that my home server is actually woefully underspeced. My desktop is close with an RTX 2070 SUPER but the card apparently doesn’t have enough RAM to successfully load the model. (Requires 16GB at least from what I’m reading although information about it is weirdly spread out)

Casually researching huggingface’s accelerate modules which apparently can split the tasks across CPU and GPU but I’m thinking the performance hit for having to swap layers on and off the GPU is going to be a hinderance. Honestly gives you some appreciation for how cost effective OpenAI is.

Going to run some more tests with the model, if it’s promising we might put some funding towards just building a wild spec’d machine to run it so I can offload a lot of random tasks to GPT-J (which would be free-ish) and save GPT-3 for the more complex tasks that GPT-J doesn’t handle well.

If anyone in the community is more versed on this sort of thing feel free to contribute, a bit out of my element but I have somewhat of an understanding.

[rd1]

Offloading basic task to GPT-J definitely makes sense. A notebook that claims to run on low vram is described here with performance metrics not sure if it is helpful

GitHub - arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram: A repository to run...

[Conqueror]

Just so happened to notice that this fired just now and checked and it’s been working extremely well apparently:

It’s been reviewing all headlines that mention ChatGPT and OpenAI and is selectively forwarding one that mention a partnership of some sort.

[Conqueror]

I’m in the process of converting the existing “test” system for Mimi’s chatbot functionality into an actual feature. This means making the system more polished so that it can be used by a wider array of functions.

/s Mode vs Normal Mode

Mimir from here on has two main operating modes. The first mode, or “Normal Mode”, utilizes a work-in-progress prompt designed to force Mimir to provide answers to any question asked of it. There isn’t a personality added to normal mode, its sole focus is analysis in as desirable a way as possible (meaning no repetitive answers, etc.)

The second mode is purely for fun. Adding /s onto the end (or anywhere in) a call to @Mimir uses a prompt designed to give us the responses we adore.

The special operations remained unchanged such as Google, image search, etc. These are just the modes for normal discussion.

Holding a conversation

The first noticeable change is that Mimir can now hold a conversation as you’d expect from services like ChatGPT. Initial implementations of this had Mimir using a shared history of the last six messages on TF, that was later moved to an individual history per user. Neither really worked very well considering too much chatter in TF would clear the history too fast when it was shared and individual history had a similar issue with the added problem that other users couldn’t build upon a conversation.

The new system utilizes Discord’s reply feature. Meaning if you reply to a message sent by Mimir, it is treated as a @Mimir call by default and automatically adds in the entire chain of replies associated with that message (up to 2500 tokens which is around ~30-45 messages iirc):

Becomes

This means that anyone can pick up on a conversation at any point by just replying to the specific message where they want to enter it at.

This function also supports both “modes” as described above. Replying to a chain where a comment was in /s mode will keep the response in /s mode.

Embed Context

To give hope to those that think all this work is to fuck off, I’ve also included the ability to use some of Mimir’s embeds for context (For those unaware, embeds are the messages that include the box with a color on it with titles, images and such.) Mimir ignores replies to embeds by default to preserve the ability to tag people in replies without triggering a message from Mimir. However the following two are exempt and do trigger a reply:

1. “Google Augmented Results”, aka Mimir’s Google lookup function.
2. News Headlines & Tweets

Replying to one of these embeds will use the content of the embed as context. For example this exchange from earlier:

In order to create the above, Mimir utilized data pulled from the first two results of a Google search. Now in order to build upon that summary, I can reply to that message and it’s used as context like so:

This used the data Mimir generated in the first query to inform the response to my question. So this provides a way to get quick context on news and such as they popup on TF.

What’s Next?

I’ve got a lot of “irons in the fire” at the moment with Mimir’s scanning functions, the positions system upgrades and such and it’s all inching forward. As for this specific project I am currently building out a vector search platform that will enable Mimir to query information that is related to a query with precision accuracy and feed that into the GPT model as context as I did above with the Google Search.

This means that SEC documents, economic data and the like will be accessible for querying using the chatbot, in essence massively expanding it’s knowledge.

The first step in this will be building out the Google Search function to use semantic search, meaning that the relevant portions of the first 5-10 pages returned by Google can be scanned and summarized for consumption by GPT instead of just throwing the full pages of the first two results in and hoping for the best.

After that is up and running I’ll be adapted to use the vector storage that I’ve set up to tap into all the data that Mimir currently has coming in, News Headlines, Tweets, SEC Documents, Fundamental Data and more.

More soon.

[Conqueror]

This functionality has taken a pretty decent step forward this weekend.

Data Vectorization & Semantic Search

Results pulled by the Google functionality are now broken down into smaller portions ran through a process called embedding. This allows us to match the smaller relevant bits to the query we’re creating ensuring that all the data shown to the GPT model is the data that we’re asking about. This replaces the old method of only grabbing the first two results and trimming them down so they fit into the GPT request. This of course returned a lot of unrelated information and potentially trimmed the pertinent bits.

The prompts are still being tweaked however, this functionality is now live and there is a noticeable increase in the effectiveness of Mimir + Google queries.

Document Analysis

You can now request analysis on specific URL’s:

Providing a query and attaching a URL afterward now directly requests that URL and provides analysis on its contents. Vector data is saved for direct requests and you can continue to query that data by replying to the “Vector Result” embed or by making another query with the same URL attached.

In the above example I asked Mimir to vectorize CVNA’s latest 8K and return information about its assets. I then replied to it’s first answer with a question about CVNA’s Master Dealer Agreement with DriveTime:

This second result was created by utilizing the saved vectors from the first query.

As I stated previously, the prompts for these functions are still very much in development however these features are live and available for testing. The saved CVNA 8K link is: https://www.sec.gov/Archives/edgar/data/1690820/000169082023000052/cvna-20221231.htm but as I stated you just have to add a URL to a query to request anything else.

The next step for this system is starting to stockpile data that can be queried against. I’m working out the logistics of that at the moment.

More soon.

[sparrow]

Great stuff dude. Looking foreward to playing around with some of these things further down the road. Definetly take the rest of the weekend off and get some rest!

[Conqueror]

Yesterday evening and today I swapped out the crawler that this function utilized with a more stealthy one that doesn’t get rejected when trying to scan news sites and the like. It also gained the ability to parse PDFs today.

The semantic search aspects seem to be working pretty well and with a better prompt suited for that task I think it’ll be wildly useful. Summarizing still needs some work as that is technically a completely different operation as it requires seeing the whole document and the way it currently works only provides it the most relevant bits.

None-the-less, this project is moving forward towards the end goal of being connected to a database of vectorized data/documents that can be queried automatically and manually as needed.

[Conqueror]

Circling back around to this. GPT-J-6B is currently running on TF as the default responder (for testing purposes).

The model is up and running on my workstation. Not great but it works.

[Conqueror]

OpenAI released their ChatGPT API endpoints today and I have migrated Mimir’s default mode over. This means that conversations with Mimir are now the same as conversing with ChatGPT.

Not all fuctions are moved as of yet, however @Mimir & Google/Vector functions are. As the GPT3.5 (ChatGPT) endpoints are way more locked down, /s will likely be staying with Davinci for the foreseeable future.

[Conqueror]

Piggy backing onto this, I’ve decided to unlock Mimir’s GPT function and it is now accessible in all channels. Google/Vector functions are still channel locked for the time being.

[rd1]

This is great Conq. I have found that models like we could get similar results out of GPT-J and Flan-T5, although does require a bit of prompt enigneering, but overall is cost efficient to do mundane things. This way reserve GPT3.5 for more complex tasks