Tap water in NZ is toxic because it contains fluoride, chlorine, aluminium (alum), and in some cases (where there are old water pipes) asbestos as well
It’s not just drinking the tap water that’s a problem – bathing and showering in it leads to fluoride and chlorine absorption too.
A good household carbon filter will take out chlorine, asbestos, and aluminium, but it will not take out fluoride. This can be done, but it requires a much more expensive filtration system (around NZ$2000-4000) to sort out your water for bathing and showering.
Reverse osmosis filters and distillers will get most of the fluoride out for drinking, but they remove the good minerals too, and this may cause other problems.
If you are unable to obtain water free from a good source, and have to buy bottled water, keep in mind that some bottled water is just tap water that’s been filtered to get rid of the chlorine taste, while some bottled spring water is good quality.
In Wellington, clean drinking water is freely available directly from the Petone Aquifer
And there is another aquifer at the Dowse Art Museum taps in Lower Hutt
For smaller amounts, and they ask customers not to take more than 2L at a time so it's not the spot for filling up a few hundred liters of bottles, there is also an artesian fountain at the Moore Wilson's store corner of Tory & College Streets Wellington.