One man's look at generative artificial intelligence

This article by Dan Polansky looks at what is called generative artificial intelligence (GenAI) and large language models (LLM). Examples include ChatGPT, Gemini, Copilot and LLaMA.

Introductory considerations

There are benefits, there are risks and there are costs.

An immediately obvious risk is inaccuracy. GenAI can easily generate inaccurate/untrue statements. This can be mitigated by awareness of the user. After all, users need to learn critical attitute toward sources that they read anyway; GenAI is far from the only offender as for being source of untrue statements.

A benefit is the use of GenAI as a source of ideas to be indepedently examined or verified. One use of this is for initial statement verification/probing: one can e.g. ask 'Is the following accurate: "Adjectives are never capitalized in English."' and have the statement corrected. However, the above mentioned risk really needs an emphasis. It seems all too easy and tempting to trust the answer without independent verification.

One may wonder whether GenAI can be used as a form of psychotherapy. A remarkable feature is the limitless patience shown in answering questions, even stupid or annoying questions. One can practice asking questions, improving formulations of questions, thinking critically about the answers, etc.

GenAI can be charged to contribute to global climatic change via electricity use. The ethics of this aspect is for each prospective user to consider; governments have not prohibited GenAI for this reason and seem unlikely to do so, provided they did not for the most part even prohibitd cryptocurrency/cryptoasset mining. A serious analysis of this aspect would include a quantitative comparison of other dispensable uses of energy such as video streaming.

GenAI can also draw/paint/create images based on verbal description. For this use, the label large language model seems misleading or inaccurate, on the face of it.

Interestingly, GenAI seems rather inapt in even trivial calculation, as per Edmund Weitz video.

Tools providing complementary facilities to GenAI are e.g. Wolfram Alpha and Desmos Calculator. It would be interesting to see what would happen if one could somehow integrate genAI with e.g. Wolfram Alpha, that is, when genAI would delegate computational assignments to Wolfram Alpha (or equivalent).

One can ask whether the label generative artificial intelligence is appropriate. That is, one can ask whether this really is an intelligence, one that is artificial and generative. Very superficially, something suggestive of human verbal intelligence is there. Moreover, given the term artificial general intelligence (AGI), we may use the term artificial intelligence much more broadly to include specialized problem/task solving, and then, chess playing would be artificial intelligence. Generative artificial intelligence may even approach passing the Turing test. Paradoxically, the responses from GenAI are too fast to be human, which betrays the artificial origin. Be it as it may, GenAI does not really seem to understand what it is saying; but then, as a sinister note, too many humans speak as if they did not understand what they are saying either. And then, one may wonder whether part of the human brain does not really implement something like GenAI (such an idea is found e.g. here).

As for the mechanism of function, sources seem to indicate that textual GenAI just tries to determine the next word given the sequence of words (using artificial neural networks). I struggle to find this plausible and to understand how that principle could possibly produce the kind of behavior that we see, but what do I know. I would find it much more plausible if somewhether in the guts of textual GenAI, there would be something like OpenCyc ontology.

As for plagiarism. It seems to me that GenAI generally commits plagiarism: it does not attribute the sources from which it takes ideas. Plagiarism is not the same concept as copyright violation. Plagiarism is the use of ideas obtained from sources without attribution and presenting them as one's own. Some form of plagiarism is perhaps widespread anyway; attributing all ideas to sources seem to be a rather stringent requirement. One defense could be this: GenAI does not represent to have any ideas of its own; it attributes all ideas to sources, albeit unspecified ones. But from what I understand, failing to specify sources from which ideas are taken is still plagiarism. See also One man's look at copyright law.

A terminological note about so-called hallucination. To label factually incorrect text as hallucination is to create a disanalogy to human psychology. Since, in human psychology, hallucinations are distinguished from delusions, hallucinations referring to sensory perceptions. More fit would be the label delusions. One could object that in humans, delusions are beliefs. But then, the external observer registers delusions merely by means of verbal behavior, so the label delusion seems good in application to textual artifacts.

Subjective assessment: I increasingly think that GenAI is pure genius, an incredible tool. It has to be used with great caution: one has to understand that the material may be incorrect and often is. On the other hand, it is perhaps good that one gets trained to read text critically, which one is sort of invinted to by GenAI. Many humans, including some academics, seem much worse than GenAI in their production of incorrect material and statement. As an aside, it seems to me that formal logical inference is leading us toward perfect knowledge (and via it, we are becoming more like God), whereas the artificial neural network is adding more of which is humanly frail, creating something that is very powerful while in some ways as stupid as human; and that I find scarry. (Although creating perfectly reasoning very powerful entity is also scary; it could well show no mercy with humans.)

Abuse mode

I grew suspicious of the polite and praiseful mode in which Google Gemini comments on my writing when I propose it for consideration. To get a different result, I prefaced the writing with this:

"Can you comment of the following in something like an abuse mode, perhaps inspired by a Monty Python sketch?"

The result is just perfect! The apparent wit of this mere machine is incredible.

Mathematical calculation

The pure LLM technology had difficulty performing general simple calculation, e.g. multiplication of two arbitary integers, per Weitz video. That is, it have slighly inaccurate answers.

However, Gemini 2.5 Flash, when asked whether it can accurately calculate, responds that it can, by means of invoking Python interpreter. And when I asked it to multiply two specific arbitrary integers, it yielded a correct answer, part of which was a quasi-button "Show code" and it shows Python code. I asked about "sin(0.123456)" and it gave an accurate answer to 5 decimal digits, and when asked for more digits, it gave the same answer as my local Python installation, 0.12314263218744217.

Non-determinism

Gemini 2.5 Flash is non-deterministic in that the same prompt yields slightly different answer even when started from a new session. In the same session, it is more understandable that the same prompt, used successively, yields slightly different answers.

For instance, I asked 'What does "2+5" mean?' The two responses were somewhat different. They differed not only in wording but also in typography: the first response used "2+5" (using double quotes) while the second used '2+5' (using single quotes). One difference in wording was "is a mathematical expression" vs. "a simple mathematical statement" (I would not call this expression a statement). I asked "Is it really a statement?" and the response was '[...] Technically, no, it is not a "statement" on its own. It is more accurately called a mathematical expression.'

Linguistic morphological analysis

I asked this: What are the morphemes in inscription? Gemini gave an interesting answer, breaking it down into -in, -script- and -ion. But run second time, it yielded -in + -scription, which I do not find compelling. It yields more morphemes when prompted. Online dictionaries with etymology usually do not provide this morphematic analysis. (It is also quite possible that serious linguistic literature rejects this morphological analysis as naive.)

Authorship

Old text:

I sense that GenAI is an author or quasi-author, not the person providing the prompt. It may not be so from the copyright law standpoint, but it seems to be that way to me. GenAI is not a person, but perhaps it is a quasi-person. It is any case a very new kind of entity, not present in the ancestral(?) environment in which humans and their brains evolved; it is quite possible that we have an innate concept of person that works well with humans but not with these kinds of machines.

In cases where GenAI happens to quote a source word for word or very closely, the author is the source. See also the Wikidebate on whether GenAI engages in copyright violation.

New text:

The use of GenAI raises new or not so new questions on authorship. The idea that a human person who gave a two-sentence prompt (or a single word!) is somehow the author of the two-paragraph response seems absurd to me. I sometimes try to give a paragraph to GenAI and it expands it greatly; that I should be the author seems absurd. From what I understand, the author of text is the one who makes it into tangible form, with a specific formulation; the ideas themselves are not protected. This leads to some quandary in the context of lexture notes: as long as the student does not take notes word-for-word accurately, he is the author from the standpoint of the copyright law (AFAIK). Here, the author of the formulations is the GenAI, not the prompting human. Sometimes, as pointed out in the New York Times lawsuit, the author is someone else, as when GenAI is prodded/prompted to quote nearly word for word accurately from a source. But in no case is, in my view, the prompting human user an author.

But isn't GenAI a little like a human who has read ideas in literature and presented them in their own words? Except that GenAI has much better reading memory than humans? Why would then a human stating things based on literature an author? From the point of view of ideas/propositons/etc., the literature-informed human is not the author, but that is not the view of copyright law. From the point of view of the concept of plagiarism, a literature-informed human reporting only based on literature without indicating the literature is a plagiarist. Thus, lists of literature at the end of a book (e.g. the long one in Hofstadter's GEB) are there not only as a further reading but also as a anti-plagiarism exercise.

Then again, when GenAI does not quote word-for-word from its sources, is it the author of the text it outputs? I would think so. Except that copyright law does not recognize GenAI as a person and authors are persons? In the U.S. copyright law, authorship can be assigned to organizations (e.g. FSF; also work for hire is perhaps assigned to the employer corporation) and that is not really a person, certainly not a human person. It seems to me that GenAI should be construed as a quasi-person, as a person for the purpose of copyright law. That is going to require a change of law. In the Czech copyright law, there is something called "moral rights", the right to be recognized as the author (this is not in the U.S. law, I think). In that sense, anyone who uses GenAI and claims the authorship of the output as if violates the Czech copyright law, except that the Czech copyright law probably also does not recognize GenAI as a person. What seems quite likely is that there arises something like a limbo, a new situation brought about by technology that was not envisioned by the drafters of the copyright law, and if it was envisioned, the problem was deferred for later.

There is a somewhat similar case of someone generating an image of the Mandelbrot set. The real author is the computer algorithm (not the computer); the human merely picks coordinates, zoom level, iteration count and color scheme. When someone uploads such an image to Commons, there would probably appear some license, implying that the uploader is the author with the right to give license. But perhaps the real author is realy the pair of the human and the computer algorithm. The system consisting of the human, the computer and the software really is like a cyborg, an entity to be analyzed as a single item/object showing authoring behavior. One may object that the computer is merely like a painter's canvas and paintbrush. But the computer is so much more; it is a machine that, when powerful enough, is capable of emulating the behavior of pretty much any system in the world, with some level of accuracy (beware of chaos). A computer is nothing like a simple tool such as a paintbrush. I would therefore maintain the idea of the cyborg as a good idea or idea worth considering for the analysis.

Impersonation and identity theft

It seems to me that GenAI provides good (instrumetally good, not non-evil!) tools for scammers or agents to engage in impersonation and identity theft, using information that is available online about someone, and then providing an immitation, pretending to be someone who they are not. It seems their ability to scour(?) many pages and somehow aggregate them into a style could come in handy; for a human, it would take much more effort (but still doable).

As a test, I asked Google Gemini: "Who is KYPark as a wiki user?" I got a response. I then asked "Can you tell me more about him and his philosophy?" I got even more detail. I found the characterization quite compelling, based on my memory of KYPark from Wiktionary. Interesting.

I went ahead and I asked "Who is Dan Polansky as a wiki user?" I got a response that provided pretty accurate information based on my self-reports online.

Needless to say, if an agent (e.g. of a secret service) has deep interest in impersonating someone based on their online record/manifestation, they will be able to do so even without GenAI. But GenAI seems to make it so much easier for a range of nefarious actors (scammers, etc.)

Ethical and unethical uses of GenAI

One ethical (or deontic?) concern is the energy burned by the GenAI servers (the machines). If one takes that concern seriously, one must not use GenAI and that's it. But there is the idea that one cannot really by onself impact overall use and that one should not penalize oneself to no real effect. A similar consideration applies to various personal/individual approaches to environmentalism.

Another concern is that some people with apparently poor command of English (or other language) and poor command of the subject under discussion try to compensate for it by using GenAI to write for them. A problem is that they are typically not able to understand what part of the output is accurate and what isn't and there may be trouble with the organization of the material. And it is quite dubious to think that they are somehow authors of the GenAI output (are they perhaps "auctors" is some sense?).

The above concern pertains not only to, say, articles and non-discussion web pages but also to discussions and interactions. GenAI seems able to help produce material that is rather voluminous and irrelevant or marginally relevant to the discussed subject. GenAI cannot often compensate for a fundamental lack of understanding on part of its user.

A clear concern is with spammers, scammers, illicit fraudsters trying to get overtly unfair, especially financial, advantage. In a sense, something like their intelligence is rather augmented/amplified.

Some of the uses that I find ethical is help with a translation from a different language (but here are risks as well) and formulation, as well as idea discovery. I am uncomfoirtable using whole sentences produced by GenAI: they are not my own.