Published: 16/02/2023 By Oliver Chapple
I have done some initial, preliminary shall we say, testing of ChatGPT's AI using an account that I recently created and using it's very own Chat example. What is clear to me is that this is a pivotal moment in technology. A major breakthrough in Natural language processing and answering you with a great degree of linguistic fluency. It's a true game changer, one that is still very much embryonic, but only if it can get better and learn to give you the right answers, which so far it doesn't seem to be able to! At least my experience so far is that its answers to generic questions, whilst incredibly well written and easy to read, as they are indeed composed in superb natural language, are often, either in part or wholly, sadly inaccurate! Just like the ill-fated Bard project and advert that Google misjudged only a couple of weeks ago in response to Microsoft's ChatGPT announcement, where Bard's wrong answer, which should have been the JW telescope, was a grossly incorrect one with what should have been a fairly simple question to answer, and was there for all to see costing Google a massive $120bn drop in its valuation overnight! Granted I have given ChatGPT some tricky questions to answer, but isn't that the whole point? ChatGPT can suffer from the same problem of getting the answer wrong just like Google Bard. The makers of ChatGPT just haven't said it can do something that it can't, at least publically and breaking trust is a very hard thing to come back from! So Bard is probably already dead, at least in name! Not tech!As far as I can see, ChatGPT's endemic and existential problem, as it stands today, is that you cannot seem to correct its answers even by giving it a helping hand at it, even though it politely thanks you for your effort in trying to do so, and for me this is the core problem with ChatGPT v3, since at present this is where ChatGPT fails for me at generic search like a search engine would need to perform, and it does fail quite dramatically. But I don't think this is an abject failure, since this problem can be overcome and you have to look at what really is working very successfully here, and that's its ability to communicate with you fluently. So it's not by any stretch a failure, at least not yet at generic search, and it's got plenty of runway and has the time to grow up and its natural language AI responses becoming beyond integral to and in our daily lives. Its AI can do so much more than simply chat to you and this is where the technology has so many other uses and advantages. But today I am simply reviewing its core potential, its current strengths and weaknesses for it to be realistically usable and a possible replacement to Google's generic Web Search engine, but in a whole new way, by "telling" you the answers you are looking for, without you having to sift through reams of information and instead helping you get right to the final answer, in a fluent, reasonably concise and a. uniquely interesting coversational way.
By way of example, it will not tell you necessarily what its source(s) are for its answers to a question you give it, therefore you cannot seem to help it get its 'researched' answers correct or corrected. Furthermore, if you try to help it get its answers right, it fails in that process, as it may suggest to you to refer to sources it won't give you when you ask it for its sources for an answer it has given you. Then the other issue is that it cannot seem to continue refining it's answer as you try to inform it of its error to narrow down the answers. I use this technique simply as the simplest way I could think of and kind of Webdadi 2.0 - Search and refine which is the way to get to a more specific result! The latter failure appears, in part at least, to result in failure as It can easily get itself into an answering loop. ChatGPT easily loses where it is in the answer refinement conversational process. The original question that has been usefully refined by it at first, after you help it to get its answer more accurately, just gets lost, and ChatGPT will simply go back to the unrefined version of its earlier answer, from your original question. Thus it forgets any subsequent 'learning', you have given it, your training so to speak, the improvement upon its first answer, and so ChatGPT today just gets its knickers in a twist!
To my mind, ChatGPT is too embryonic yet to use straight out of the box for say a genericized search engine, but maybe there are some great and real-world opportunities for it, I'm sure people can and will share how they have managed to box ChatGPT into working really very well at returning certain information and in certain formats, very accurately and every time. However, I would imagine at this stage this would only be for the information of their own curation. It seems at the moment to me at least, that ChatGPT thinks it knows the answers to things and with certainty when it actually doesn't!
So whilst there is a lot of room for improvement with this technology, it has, to its huge credit, incredibly all but rendered the likes of traditional Natural Language Processing tech, like Dialogflow, as technological dinosaurs already. You no longer need to train a chatbot, to have a conversation with you, or answer you in a natural way by giving it stock answers written by you, but you or a community do have to give it the right answers and the right information, the proverbial facts as it cannot necessarily create facts from its own research! Sadly at present, to many questions that you can ask it today, you'll just get the wrong answers from ChatGPT. I firmly believe however, this current status quo will, I think without a doubt, will massively improve, as the bot and its AI is improved and trained over time to allow for correction and updating its knowledge or how it learns and interprets information. With the investment that Micorosft has made into ChatGPT, and what this could do for Microsoft if they get it right, will be their Google denouement. It's a historic moment indeed, like Google's first search offering to the world in 1998. But I do believe if Microsoft, finds a way to get this technology to be taught and to self-learn accurately, then this AI tech will probably, in my view, make Google's life in the short term at least, very difficult, and this tech is surely the Internet Search panacea of tomorrow.
Something tells me that the way Wikipedia has evolved to curate accurate content is perhaps the example and signpost as one way to go to achieve this outcome for ChatGPT, as Wikipedia is real-world evidence that it can be achieved, when it's done by the right community. So Microsoft will have to rely on its own community to train its dragon. How this then will work for the business community, such that its knowledge is not tricked and is accurate, remains to be seen and a way off. Perhaps it will be tricked initially as people tricked Google in the early days, by businesses stuffing keywords in their Websites to get listed at the top, but this AI tech is the future folks, at least in terms of AI chat with natural language processing. Mark my words or I'll eat my hat!