Axel Højmark


2023

pdf bib
Formalizing content creation and evaluation methods for AI-generated social media content
Christian Jensen | Axel Højmark
Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)

This study explores the use of large language models (LLMs), such as ChatGPT and GPT-4, in creating high-quality text-based social media content for businesses on LinkedIn. We introduce a novel architecture incorporating external knowledge bases and a multi-step writing approach, which extracts facts from company websites to form a knowledge graph. Our method’s efficacy is assessed using the “Long-LinkedIn” evaluation dataset designed for long-form post generation. Results indicate that our iterative refinement significantly improves content quality. However, knowledge-enhanced prompts occasionally reduced quality due to potential formulation issues. LLM-based evaluations, particularly using ChatGPT, showcased potential as a less resource-intensive alternative to human assessments, with a notable alignment between the two evaluation techniques.
Search
Co-authors
Venues