I seen a documentary not that long ago on national geographic about some of the large lithium mines that operate around the world.
The conditions were atrocious in some of them, not to mentioned all of the by-products used to refine the lithium from rocks into batteries. Stuff like acid baths etc.. produce a lot of waste.
Not to mention that unlike petrol that burns quick and fast, lithium likes to both burn and can be explosive in the presence of water and heat.
Personally I'm a fan of nuclear power, which i think is very under-invested in across the UK because people associate it with both weapons and places such as Chernobyl etc..