Definition of the heuristic way
From the Greek for 'discovery'
I have chosen to call the successful methodology 'heuristic' rather than 'scientific' in order to separate it from commonly-held ideas about what its proper subject-matter is.
The word 'heuristic' comes from the ancient Greek 'heurisko' (to find). You may remember Archimedes' famous cry, "Eureka!" ("I have found it!") when he jumped out of his bath after having thought of Archimedes' principle.
Negative circular processes are involved
The most well-known feature of the scientific method—which we are now going to call the heuristic method—and one which relates it directly to the fundamentals of cybernetics, is that it uses circular processes. What is more, as Popper pointed out, these processes characteristically work with negative instances, rather than with positive ones.
The most important circular process that it uses is an alternation between logical deduction within a model, and empirical testing of the result against reality. Note that there is no stopping state, the match of model and reality is achieved only as long as no negative instances are known, but you cannot assert that there are no negative instances and are obliged to keep looking for them.
Giving up the certainty of success
The other most striking characteristic of the heuristic method follows directly from the first. It is the giving up of the search for certainty in return for a greater chance of success.
Although this characteristic of the scientific method may be less well known than the circularity of the processes which it uses, it is the main reason for the spectacular success of the method.
Heuristics in logic
In logic, a 'heuristic approach' to a problem is one which uses successive evaluations of trial and error to arrive at a final result. This may solve a problem but offers no guarantee of success.
Heuristics in computing
In computing, what is called 'heuristic programming' solves a problem by a method of trial and error, in which the success of each attempt at solution is assessed and used to improve the subsequent attempts, until a solution acceptable within defined limits is reached. For our purposes, the most important point about heuristic programming is that it involves giving up the certainty of achieving a solution, as would be the case with an algorithmic program, and in return taking advantage of the possibility of achieving a much better solution.
The search for certainty
It is this giving up of the certainty of proof, in return for a greater chance—but only a chance—of finding a solution, that is the most important feature of the scientific method, and the reason why I have chosen to call it 'the heuristic method'.
Historically speaking, the main methodological discovery of science was that the search for certainty, so dear to human beings, never achieves progress; whereas never being satisfied with what you have achieved so far, but always striving to improve understanding by further discovery, does result in real advances of knowledge.
The idea that a method exists for achieving certain knowledge goes back at least as far as Euclid (c300 BC), whose method of logical construction from self-evident postulates worked well for geometry, and seemed to be a model for finding out about the rest of reality.
Since then, there have been many attempts to build up a structure of certain knowledge by deduction from self-evident premises.