The trigger began with a single piece of feedback
I was developing an AI system that automatically generates medical articles, and one day I received harsh feedback from test users.
“The main text is full of bullet points, and I can't use it at the current quality. I want you to write sentences that are more natural and make you want to read on.”
Certainly, when I read the generated article again, it was just hard sentences like textbooks. Every time statistical data was shown, it was repeated “according to ~” and “in the survey of ~,” and the content made me tired while reading it. Furthermore, symbols such as “:” and “｜” were abused in the title, giving a somewhat unnatural impression.
This wasn't just a question of writing style; it was my own design mistake that didn't allow AI to correctly understand “what is a good article.” From there, a full-scale improvement project will begin.
Two conflicting demands for medical articles
In writing medical articles, we constantly face conflicting demands. One is “accuracy.” Since medical information is directly linked to human health, vague expressions or unfounded claims are not permitted. But at the same time, there is one more requirement. It's “readability.”
No matter how accurate it is, no one will read until the end if the sentences are full of technical jargon and hard. In particular, for people who come to read the article with anxiety about their health, the list of difficult medical terms only causes further anxiety.
How do you balance these two? That was the most troubling issue with this project.
A “double-edged sword” called evidence
To increase the reliability of medical articles, I built into the system the ability to automatically gather data from 4 sources. The first is a medical paper database provided by the US National Institutes of Health. The second is health statistics published by Japan's Ministry of Health, Labor, and Welfare. The third is the government statistics portal site. And the fourth is international statistics from the World Health Organization.
If you include this data in an article, you can certainly provide “evidence-based information.” But there was a catch here. AI was too serious.
When I read the generated article, sentences like this were lined up.
“According to a survey by the Ministry of Health, Labor, and Welfare, diabetics are on the rise. According to the WHO report, similar trends are being seen globally. In the National Health and Nutrition Survey, lifestyle improvements are needed.” It is accurate. But is it fun to read? It's as stiff as an academic paper, and readers will leave halfway through.
Change of thinking: having the evidence focus on “behind the scenes”
Here, I made a major shift in my thinking. I realized that evidence should be treated as something “supported behind the scenes” rather than something “put forward.”
If you compare it to cooking, it's like until now, all the production areas and nutritional information lists of ingredients have been placed on top of food. The information is certainly accurate, but it makes me lose my appetite. Originally, it's a good idea to write that kind of information in a small amount “under the menu.”
Therefore, the instructions to the AI were drastically changed. “Let statistical data blend naturally into sentences. Limit quotes to about 2 to 3 times for the entire article. Detailed sources should be summarized and listed at the end of the article”
Due to this change, the article has changed like this.
“Actually, in Japan, about 20 million people have the possibility of diabetes. This is definitely no one else's problem. But that's why it's important to know prevention methods you can start today.”
It conveys the same information, but I think it has changed to an expression that resonates with the hearts of readers.
What I needed to “win” in Google searches
As we proceeded to improve the system, another issue came up. It's a question of “how to differentiate it from other articles displayed at the top of searches.”
No matter how good an article you write, it's meaningless if no one reads it. And many people
The reality is that I only read the top 5 Google search articles. In other words, if you don't know what kind of content the top articles deal with, let alone differentiate yourself, you can't even stand at the starting line of competition.
So we decided to add a new feature. It is a function that actually performs keyword searches on Google and automatically analyzes what the top 5 articles write. We decided to let AI read the title of each article, a summary of the content, and even the search intent of “what are readers looking for.”
With this function, AI can now automatically find “points not mentioned in competitor articles” and “topics that should be dug deeper.” It is now possible for AI to do its own market research and develop strategies to write winning articles, so to speak.
Unexpected technical trouble and learning
The development seemed to be going well, but an unexpected error occurred during the system startup test. The program misinterpreted the string and caused a syntax error. The cause was a special symbol I used in my instructions to AI. In the world of programs, certain symbols have special meanings. I didn't take that into consideration and ended up using symbols as if I were writing a normal sentence.
It actually took the longest to resolve this issue. I reread the error messages over and over and checked what was wrong with the code one by one. Finally, the problem was solved by removing all problematic symbols and rewriting them to simpler expressions. What I learned from this trouble is “the importance of being simple.” Complex mechanisms are also just as easy to break. I realized once again that a really good system is something simple yet powerful.
What the completed system can do
As a result of repeated improvements, the system is now able to generate articles in the following flow.
First, it automatically analyzes the articles that appear at the top of Google searches. Next, we gather the latest research information from medical paper databases around the world. In parallel with that, we obtain statistical data from the Japanese government and international statistics from the World Health Organization.
All of this information is integrated, and AI will weave it up as a single article. At this time, it is written in accordance with the instructions of “natural sentences,” “keep quotes modest,” and “add storyline,” which I mentioned earlier.
Once the article is complete, another AI will check medical accuracy one last time. It is a mechanism that checks whether there is any dangerous information, whether there is any expression that violates the law, and whether necessary notes are included, and issues a warning if there is a problem.
This process takes only about 20 to 30 seconds. If humans try to do the same thing, tasks that take hours can now be completed instantaneously.
Where are we headed next
Through this project, I experienced firsthand the difficulty and fun of “making AI do something.” AI faithfully executes what is instructed, but depending on the “way instructions are given,” the results will change as much as heaven and earth.
Generate articles that are easy to read while maintaining accuracy in the sensitive field of medical information. At first glance, these issues seemed contradictory; in fact, they were not. Rather, I realized that because it is really accurate information, I have a responsibility to convey it in an easy-to-understand manner.
In the future, we plan to add functions for visualizing statistical data with graphs and the ability to translate articles into multiple languages. Above all, I hope this system will reduce the burden on health care workers and help deliver accurate health information to more people.
Technology is just a tool. What matters is what you use it to achieve. Going forward, I would like to continue development while facing these questions.
