This may well be very useful, but Claude has ruined me and I have a visceral reaction to ‘gates’ in the opening sentence and couldn’t get any further. If a tool is worth building and sharing, surely it’s also worth writing a brief summary of what it’s supposed to achieve in your own words?
Fixing unreliable agents by adding more unreliable agents. It's a shame we can't do something about the tendency of LLMs to give more weight to their weights than to the data they're analysing.
I guess it all comes back to the fact that they don't have a clear division between data they have learned about the world, instructions and data under analysis.
You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.
> Fixing unreliable agents by adding more unreliable agents.
Depending on how you combine imperfect systems, you can lower or increase error rates.
> data they have learned about the world, instructions and data under analysis.
I don't think there really is a clear distinction between these things though.
And I don't mean this in a gotcha way - these things apply to people too. Not that that means they're like people, but we have successfully built enormous systems out of fallible people, and other imperfect things.
The goal is rarely "eliminate all errors" but "have a thing that's useful, given a possibility of things going wrong".
Depending on how you combine imperfect systems, you can lower or increase error rates.
Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention. They are not reliable in the way we're used to considering and don't work well without supervision from careful humans at present.
I don't think there really is a clear distinction between these things though.
Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
I do think this is a significant problem specific to LLMs though (that I at least see when using them) - we don't have a way to constrain them in what knowledge they consider. This can be a bonus (they are good at connecting faraway bits of maths in interesting ways), but it can be a significant problem when asking for a summary of a data - confabulations and data from outside the context easily leak into the output of the LLM. That is undesirable behaviour and we have no reliable way to constrain it.
I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
> Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention.
Ah but the point here IIUC is moving from "let an LLM figure out the right way of querying the data and what you mean with your terms from a wide span of different documents and sources, which it has to perform multiple searches on" to "let an LLM figure out the right way of querying your data with a coherent and machine readable ontology".
While the conversion from "load of data" to "defined structure" is error prone, you now have the second step operating on the same converted thing, and you have ways of fixing that over time (the corrections).
By splitting the task into two distinct phases, you can both interpret and fix the middle step, and remove a large class of mistakes and/or differences caused by different interpretations of the same documents.
There are really two answers to it IMO that this boils down to:
1. This process may be less error prone for that first step than the LLM you're using. That could reduce errors.
2. This process means the first step is run once rather than every time which means you cut out one area where there can be differences each time.
> Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
You're right, I think there are useful groupings, but I think any effort to fully isolate them is doomed to fail because they inherently blend. Your instructions contain things that can only be understood in context of world knowledge and the data. The data only makes sense with certain bits of world knowledge.
We also know that these things happen with humans (again just to pre-empt how others may read this and not saying you are accusing me of this, I'm not gotcha-ing here) and so we often have to be careful about how we ask questions, how different cultural backgrounds impact understanding and so forth. We have to take care about how we structure things.
> I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
I think this effort does address that. A really key thing is that this is about turning human readable questions into SQL that then only query the actual data and you see the result of that query - it allows you to constrain things more (at both points).
I don't understand this ontology thing. What's the difference between this and a memory system plus skills to collect your enterprise's APIs and CLIs etc?
Ontology is a so-called white-box approach for AI: all dependencies are clearly stated and formalised. If something does not work as you wish, unlike LLMs, in ontology world you can clearly diagnose where the system fails. I understood the NVIDIA's idea this way: we take unreliable LLM and make a "reliable" ontology using it.
If you look back into history, Prolog , was exactly a sort of "memory system" or, better put, a human-friendly database. Ontology is a further development of this concept.
One issue in a business is what does all the data actually mean? What are the various definitions and how do they relate to each other. What's a customer? Should that include people that have bought things, or anyone who is potentially buying? Do different systems use the same definitions, what are the columns, etc.
An ontology in this kind of context means a more precise machine readable spec of this and how it all relates to each other. You can totally build one by hand. The idea here is I think that it tries to figure it out itself and can take corrections over time, so you can ask natural language questions and get more reliable results rather than hoping an llm works out what "customers this month" means the same way each time over a wide span of data rather than having a much clearer API to work against.
A memory system is managed by the RL trained behavior, these systems use specific prompts and ordinary software to hardcode a policy. The RL version is usually more efficient, but it can't be improved or made more repeatable.
How about we start by open-sourcing the NVIDIA user-space libraries: libcuda, libcudart, libOpenCL, libnvtx, as well as nvidia-smi and the CUDA and OpenCL profilers.
The readme seems to be written for enterprise project managers. Does anyone know what philosophy or academic ontology work this ties to, if any? I’m having trouble understanding the specifics, I’ll be honest…
I guess it all comes back to the fact that they don't have a clear division between data they have learned about the world, instructions and data under analysis.
You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.
Depending on how you combine imperfect systems, you can lower or increase error rates.
> data they have learned about the world, instructions and data under analysis.
I don't think there really is a clear distinction between these things though.
And I don't mean this in a gotcha way - these things apply to people too. Not that that means they're like people, but we have successfully built enormous systems out of fallible people, and other imperfect things.
The goal is rarely "eliminate all errors" but "have a thing that's useful, given a possibility of things going wrong".
Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention. They are not reliable in the way we're used to considering and don't work well without supervision from careful humans at present.
I don't think there really is a clear distinction between these things though.
Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
I do think this is a significant problem specific to LLMs though (that I at least see when using them) - we don't have a way to constrain them in what knowledge they consider. This can be a bonus (they are good at connecting faraway bits of maths in interesting ways), but it can be a significant problem when asking for a summary of a data - confabulations and data from outside the context easily leak into the output of the LLM. That is undesirable behaviour and we have no reliable way to constrain it.
I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
Ah but the point here IIUC is moving from "let an LLM figure out the right way of querying the data and what you mean with your terms from a wide span of different documents and sources, which it has to perform multiple searches on" to "let an LLM figure out the right way of querying your data with a coherent and machine readable ontology".
While the conversion from "load of data" to "defined structure" is error prone, you now have the second step operating on the same converted thing, and you have ways of fixing that over time (the corrections).
By splitting the task into two distinct phases, you can both interpret and fix the middle step, and remove a large class of mistakes and/or differences caused by different interpretations of the same documents.
There are really two answers to it IMO that this boils down to:
1. This process may be less error prone for that first step than the LLM you're using. That could reduce errors. 2. This process means the first step is run once rather than every time which means you cut out one area where there can be differences each time.
> Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
You're right, I think there are useful groupings, but I think any effort to fully isolate them is doomed to fail because they inherently blend. Your instructions contain things that can only be understood in context of world knowledge and the data. The data only makes sense with certain bits of world knowledge.
We also know that these things happen with humans (again just to pre-empt how others may read this and not saying you are accusing me of this, I'm not gotcha-ing here) and so we often have to be careful about how we ask questions, how different cultural backgrounds impact understanding and so forth. We have to take care about how we structure things.
> I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
I think this effort does address that. A really key thing is that this is about turning human readable questions into SQL that then only query the actual data and you see the result of that query - it allows you to constrain things more (at both points).
[0] https://mirekrusin.com/cave
An ontology in this kind of context means a more precise machine readable spec of this and how it all relates to each other. You can totally build one by hand. The idea here is I think that it tries to figure it out itself and can take corrections over time, so you can ask natural language questions and get more reliable results rather than hoping an llm works out what "customers this month" means the same way each time over a wide span of data rather than having a much clearer API to work against.
How about we start by open-sourcing the NVIDIA user-space libraries: libcuda, libcudart, libOpenCL, libnvtx, as well as nvidia-smi and the CUDA and OpenCL profilers.