Of course not. There are very good reasons to keep informal methods in the toolkit, rather than replacing them wholesale by more sophisticated methods. Informal methods have always played an important role in science. Since the goal of a scientific argument is to convince one's peers, the degree of formal rigor deemed necessary depends a great deal on the standards conventional in the community. Many firmly established findings were first published in papers that would never pass peer review by today's standards. Even thought experiments, the most "informal" type of experiment imaginable, can be highly convincing, merely by highlighting the theoretical implications of everyday observations. Much linguistic argumentation has the character of thought experiments, since native speakers are invited to test claimed judgments against their own intuitions. This ease of testability means that the results of a typical linguistic "experiment" are replicated far more often than results in fields with more demanding methodological protocols. Finally, there is the issue of limited resources: carrying out rigorous tests of every intuitively obvious empirical claim would be a serious waste of time and energy.

Nevertheless, linguists should understand that the complex methodological protocols of the other experimental cognitive sciences are not caprice, but developed over a history of almost two centuries in the face of very difficult empirical challenges: the mind is a black box, and behavior is messy. Testing hypotheses about the mind using only physical (mostly behavioral) evidence is so difficult to do right that the behaviorists gave up entirely, and even today's cognitive psychologists and neuroscientists retain a high degree of skepticism about mentalist claims that are not backed up by models that explicitly link behavior to the mind. Linguists have highly developed models of mental entities (elements of grammar), and they recognize that competence (mind) and performance (behavior) are not the same thing, but they generally have no interest at all in developing explicit competence-performance linking models. The result is that they have no way to tell when evidence that seems intuitively obvious is actually anything but: tainted by bias, or simply wrong. A psychologist might say that this slippery-slope problem calls all linguistic judgment data into question. However, such a response not only dismisses the arguments in the previous paragraph without any real counterargument, but it is also rather hypocritical, given that "all experiments leak": the discussion sections of the typical psychology paper have just about as much speculation and special pleading as the typical linguistics paper.

The most fundamental reason why experimental syntacticians follow the psycholinguistic practice of testing sentences individually is that this best reflects the nature of the hypotheses they are testing. That is, syntax is about individual sentences, not sentence pairs, so the testing unit should be the sentence, just as discourse hypotheses should be tested with strings of sentences. Even Optimality Theory, which relies on a comparison mechanism to make predictions about the grammaticality of forms, is not, strictly speaking, about comparison as an empirical subject matter.