I have changed the group description to: Literary fiction is a term used in the book-trade to distinguish fiction that is regarded as having literary merit from most commercial or "genre" fiction. Unlike genre fiction plot is not the central concern.
For this I pinched from Wikipedia [https://en.wikipedia.org/wiki/Literary_fiction] which I recommend reading.
In there is also the previous sort of description for this group: A concern with social commentary, political criticism, or reflection on the human condition that is only so good as it means, for example, including the trials of a drug addict in fiction alone does not make that book literary fiction.
The question-begging term is "literary merit," of course.
Briefing for a Descent into Hell by Doris Lessing - Yes
Harry Potter and the Sorcerer's Stone by J.K. Rowling - No
I would use the primary distinction as the (un)importance of plot as primary concern and not any relevance of social commentary.