Previous winners AS Byatt and JM Coetzee are among the finalists for the prestigious Booker Prize for fiction.