If I remember the correspondence correctly, Byron was a bit bemused by the correspondence with a lady people called a "pattern card of propriety." I think Miss Milbanke thought of herself in that manner, as well. For instance, when invited to a country house for a visit and informed that Lady Holland( a divorced woman) would be present, she is reputed to have said something to the tune of she believed her own reputation would protect her from being besmirched by the presence of Lady Holland.
In Sense and Sensibility , Jane Austen has characters say and act on the belief that a couple who are exchanging letters are engaged to each other.
If Jane Austen is correct, then when Byron proposed to Miss Milbanke, her refusal must have come as a shock to him for she had given him every indication of interest -- though her letters are quite proper and mainly discuss poetry and some "poets."
It is difficult to fond any one who looks at Byron's life in an impartial manner. Everyone seems to look at his life from a predetermined platform. Certain interpretations are taken for granted and alternate ones aren't often considered. Byron's efforts to rebrand himself were all too successful.