This is where harness, and the fact that a machine can be endlessly prompted to try again comes in.
Even if an LLM starts by pursuing things that follow human bias, continuous failures and re-prompting to try something different will eventually force it to consider things outside of what ever biases it has.
You can do the same thing to a human. But most people would consider it unethical to lock someone in a box and force them to keep trying to solve the same problem over and over again until they figure it out.
Your comment stopped me in my tracks a little bit.
Is a 'bias' in a piece of writing generally a property of word to word choice and sentence to sentence construction or is it something more nebulous? Especially in terms of the appreciation of mathematics and someone's hesitance about publishing a mathematical argument they think is ugly or brute forced in some way.
> Is a 'bias' in a piece of writing generally a property of word to word choice and sentence to sentence construction or is it something more nebulous?
I wouldn't have expected that there was post training specifically on the issue of looking for proofs vs counter examples. But it might be that other post training has a side effect of making AIs better at looking for counter examples. I wonder if these agents are overall less biased and more rational than humans. Can you expand on what you mean by goal based training?
