"Samurai Swordsman: Master of War," by Stephen Turnbull (of course.) This is a general history book; imagine reading an Osprey book, but 200 pages. What I find interesting about the book is its smaller discussion about major battles to make room for more discussion about samurai fighting outside of major warfare.
In it you'll find discussion about Samurai in society, Sword schools (and their sometimes violent transitions,) Sword pilgrimages (musha shugyo) by the likes of Musashi and Bokuden among others, Street gang warfare (my favorite chapter), Samurai during the Tokugawa peace through the Meiji Restoration. There's even a chapter covering samurai women.
Among the chapters, my favorite sections (so far) include an interesting (if short) discussion about Edo firefighters battling one another;) a few accounts of samurai vendettas including one that spanned almost 30 years; and the chapter about street gangs in the Edo period (this chapter contains the firefighter section.)
There are indeed a few chapters on major warfare, and Turnbull gives a nice discussion on the evolution of the samurai from mounted archer to swordsman (via the spear.) He also does a great job of giving small accounts of battle, street fights, duels, killings and vendettas. Many of the accounts are somewhat detailed (though, not blow-by-blow.)
The book is illustrated mostly with Japanese prints, but there are also a few photographs of reenactors. I would recommend the book LAST as a reference for painting minis, but first if you're looking for some good stories without getting too scholarly.