Annoying the user: Please enter numbers only

Everyone of us sometime or other had to restrict the user from entering non-numeric values in to a text field. We normally validate user input when user presses the submit button or when the user leave the input field and display the error message in a span or using alert() javascript function. However what will happen if we validate user input as the user is typing in the field and also use a alert box to display error message instead of showing it in a span?