I’d put money on humans scoring even less on subjects they’ve never heard of.
They are testing is the ability to reason. The AI, or human, can still use the internet to find out the answer. Here’s a sample question that illustrates the distinction.
Hummingbirds within Apodiformes uniquely have a bilaterally paired oval bone, a sesamoid embedded in the caudolateral portion of the expanded, cruciate aponeurosis of insertion of m. depressor caudae. How many paired tendons are supported by this sesamoid bone? Answer with a number.
That reads like the sort of thing Wolfram Alpha was designed to absolutely obliterate, if only the raw data representing each of those keywords had been loaded in.
They are testing is the ability to reason. The AI, or human, can still use the internet to find out the answer. Here’s a sample question that illustrates the distinction.
Hummingbirds within Apodiformes uniquely have a bilaterally paired oval bone, a sesamoid embedded in the caudolateral portion of the expanded, cruciate aponeurosis of insertion of m. depressor caudae. How many paired tendons are supported by this sesamoid bone? Answer with a number.
That reads like the sort of thing Wolfram Alpha was designed to absolutely obliterate, if only the raw data representing each of those keywords had been loaded in.