I believe they can challenge it, but not to you, as you don’t set the BRMA .Basically they feel the ‘line on the map’ has been drawn in the wrong place, essentially?
Does this page, courtesy of shelter, help?
https://england.shelter.org.uk/professional_resources/legal/benefits/local_housing_allowance/how_local_housing_allowance_is_calculated#title-6
‘There is no right of appeal regarding the amount of the LHA rate, although an appeal can be made concerning the size criteria that has been applied to a claimant. The decision making process whereby a BRMA is drawn or the LHA rates are set could be challenged by judicial review’
