Approximation of experimental data is a method based on replacing experimentally obtained data with an analytical function that most closely passes or coincides at nodal points with the original values (data obtained during an experiment or experiment). Currently, there are two ways to define an analytical function:
By constructing an n-degree interpolation polynomial that passes directly through all points a given data array. In this case, the approximating function is presented in the form of: an interpolation polynomial in Lagrange form or an interpolation polynomial in Newton form.
By constructing an n-degree approximating polynomial that passes in the immediate vicinity of points from a given data array. Thus, the approximating function smooths out all random noise (or errors) that may arise during the experiment: the measured values during the experiment depend on random factors that fluctuate on their own random laws(measurement or instrument errors, inaccuracy or experimental errors). In this case, the approximating function is determined using the method least squares.
Least square method(in the English literature Ordinary Least Squares, OLS) is a mathematical method based on determining an approximating function that is constructed in the closest proximity to points from a given array of experimental data. The closeness of the original and approximating functions F(x) is determined by a numerical measure, namely: the sum of squared deviations of experimental data from the approximating curve F(x) should be the smallest.
Approximating curve constructed using the least squares method
The least squares method is used:
To solve overdetermined systems of equations when the number of equations exceeds the number of unknowns;
To find a solution in the case of ordinary (not overdetermined) nonlinear systems of equations;
To approximate point values with some approximating function.
The approximating function using the least squares method is determined from the condition of the minimum sum of squared deviations of the calculated approximating function from a given array of experimental data. This criterion of the least squares method is written as the following expression:
The values of the calculated approximating function at the nodal points,
A given array of experimental data at nodal points.
The quadratic criterion has a number of “good” properties, such as differentiability, providing a unique solution to the approximation problem with polynomial approximating functions.
Depending on the conditions of the problem, the approximating function is a polynomial of degree m
The degree of the approximating function does not depend on the number of nodal points, but its dimension must always be less than the dimension (number of points) of a given experimental data array.
∙ If the degree of the approximating function is m=1, then we approximate table function straight line (linear regression).
∙ If the degree of the approximating function is m=2, then we approximate the table function with a quadratic parabola (quadratic approximation).
∙ If the degree of the approximating function is m=3, then we approximate the table function with a cubic parabola (cubic approximation).
IN general case, when it is necessary to construct an approximating polynomial of degree m for given table values, the condition for the minimum of the sum of squared deviations over all nodal points is rewritten in the following form:
- unknown coefficients of the approximating polynomial of degree m;
The number of table values specified.
A necessary condition for the existence of a minimum of a function is the equality to zero of its partial derivatives with respect to unknown variables . As a result, we obtain the following system of equations:
Let's transform the resulting linear system of equations: open the brackets and move the free terms to the right side of the expression. As a result, the resulting system of linear algebraic expressions will be written in the following form:
This system of linear algebraic expressions can be rewritten in matrix form:
The result was a system linear equations dimension m+1, which consists of m+1 unknowns. This system can be solved using any method for solving linear problems. algebraic equations(for example, by the Gaussian method). As a result of the solution, unknown parameters of the approximating function will be found, providing minimum amount squared deviations of the approximating function from the original data, i.e. best possible quadratic approximation. It should be remembered that if even one value of the source data changes, all coefficients will change their values, since they are completely determined by the source data.
Approximation of source data by linear dependence
(linear regression)
As an example, let's consider the technique for determining the approximating function, which is specified in the form of a linear dependence. In accordance with the least squares method, the condition for the minimum of the sum of squared deviations is written in the following form:
Coordinates of table nodes;
Unknown coefficients of the approximating function, which is specified as a linear dependence.
A necessary condition for the existence of a minimum of a function is the equality to zero of its partial derivatives with respect to unknown variables. As a result, we obtain the following system of equations:
Let us transform the resulting linear system of equations.
We solve the resulting system of linear equations. The coefficients of the approximating function in analytical form are determined as follows (Cramer’s method):
These coefficients ensure the construction of a linear approximating function in accordance with the criterion of minimizing the sum of squares of the approximating function from the given tabular values (experimental data).
Algorithm for implementing the least squares method
1. Initial data:
An array of experimental data with the number of measurements N is specified
The degree of the approximating polynomial (m) is specified
2. Calculation algorithm:
2.1. The coefficients are determined for constructing a system of equations with dimensions
Coefficients of the system of equations (left side of the equation)
- column number index square matrix systems of equations
Free terms of a system of linear equations (right side of the equation)
- index of the row number of the square matrix of the system of equations
2.2. Formation of a system of linear equations with dimension .
2.3. Solving a system of linear equations to determine the unknown coefficients of an approximating polynomial of degree m.
2.4. Determination of the sum of squared deviations of the approximating polynomial from the original values at all nodal points
The found value of the sum of squared deviations is the minimum possible.
Approximation using other functions
It should be noted that when approximating the original data in accordance with the least squares method, the logarithmic function, exponential function and power function are sometimes used as the approximating function.
Logarithmic approximation
Let us consider the case when the approximating function is given logarithmic function type:
Least square method
In the final lesson of the topic, we will get acquainted with the most famous application FNP, which finds the widest application in various fields of science and practical activity. This could be physics, chemistry, biology, economics, sociology, psychology, and so on and so forth. By the will of fate, I often have to deal with the economy, and therefore today I will arrange for you a trip to an amazing country called Econometrics=) ...How can you not want it?! It’s very good there – you just need to make up your mind! ...But what you probably definitely want is to learn how to solve problems least squares method. And especially diligent readers will learn to solve them not only accurately, but also VERY QUICKLY ;-) But first general statement of the problem+ accompanying example:
Let in some subject area indicators that have a quantitative expression are studied. At the same time, there is every reason to believe that the indicator depends on the indicator. This assumption can be either a scientific hypothesis or based on elementary common sense. Let's leave science aside, however, and explore more appetizing areas - namely, grocery stores. Let's denote by:
– retail area of a grocery store, sq.m.,
– annual turnover of a grocery store, million rubles.
It is absolutely clear that the larger the store area, the greater in most cases its turnover will be.
Suppose that after carrying out observations/experiments/calculations/dances with a tambourine we have numerical data at our disposal: 
With grocery stores, I think everything is clear: - this is the area of the 1st store, - its annual turnover, - the area of the 2nd store, - its annual turnover, etc. By the way, it is not at all necessary to have access to classified materials - a fairly accurate assessment of trade turnover can be obtained by means of mathematical statistics
. However, let’s not get distracted, the commercial espionage course is already paid =)
Tabular data can also be written in the form of points and depicted in the familiar form Cartesian system .
Let's answer an important question: How many points are needed for a qualitative study?
The bigger, the better. The minimum acceptable set consists of 5-6 points. In addition, when the amount of data is small, “anomalous” results cannot be included in the sample. So, for example, a small elite store can earn orders of magnitude more than “its colleagues,” thereby distorting general pattern, which is what you need to find!
To put it very simply, we need to select a function, schedule which passes as close as possible to the points 
. This function is called approximating
(approximation - approximation) or theoretical function
. Generally speaking, an obvious “contender” immediately appears here - a high-degree polynomial, the graph of which passes through ALL points. But this option is complicated and often simply incorrect. (since the graph will “loop” all the time and poorly reflect the main trend).
Thus, the sought function must be quite simple and at the same time adequately reflect the dependence. As you might guess, one of the methods for finding such functions is called least squares method. First, let's look at its essence in general view. Let some function approximate experimental data: 
How to evaluate the accuracy of this approximation? Let us also calculate the differences (deviations) between the experimental and functional values (we study the drawing). The first thought that comes to mind is to estimate how large the sum is, but the problem is that the differences can be negative (For example, 
)
and deviations as a result of such summation will cancel each other out. Therefore, as an estimate of the accuracy of the approximation, it begs to take the sum modules deviations:
or collapsed: (in case anyone doesn't know:
is the sum icon, and
– an auxiliary “counter” variable, which takes values from 1 to
)
.
Bringing experimental points closer various functions, we will receive different meanings, and obviously, where this amount is smaller, that function is more accurate.
Such a method exists and it is called least modulus method. However, in practice I received much greater distribution least square method, in which possible negative values are eliminated not by the module, but by squaring the deviations:
, after which efforts are aimed at selecting a function such that the sum of squared deviations 
was as small as possible. Actually, this is where the name of the method comes from.
And now we're going back to something else important point: as noted above, the selected function should be quite simple - but there are also many such functions: linear , hyperbolic , exponential , logarithmic , quadratic etc. And, of course, here I would immediately like to “reduce the field of activity.” Which class of functions should I choose for research? A primitive but effective technique:
– The easiest way is to depict points on the drawing and analyze their location. If they tend to run in a straight line, then you should look for equation of a line 
with optimal values and . In other words, the task is to find SUCH coefficients so that the sum of squared deviations is the smallest.
If the points are located, for example, along hyperbole, then it is obviously clear that the linear function will give a poor approximation. In this case, we are looking for the most “favorable” coefficients for the hyperbola equation 
– those that give the minimum sum of squares 
.
Now note that in both cases we are talking about functions of two variables, whose arguments are searched dependency parameters:
And essentially we need to solve a standard problem - find minimum function of two variables.
Let's remember our example: suppose that “store” points tend to be located in a straight line and there is every reason to believe that linear dependence turnover from retail space. Let's find SUCH coefficients “a” and “be” such that the sum of squared deviations 
was the smallest. Everything is as usual - first 1st order partial derivatives. According to linearity rule You can differentiate right under the sum icon: 
If you want to use this information for an essay or term paper, I will be very grateful for the link in the list of sources; you will find such detailed calculations in few places: 
Let's create a standard system: 
We reduce each equation by “two” and, in addition, “break up” the sums: 
Note
: independently analyze why “a” and “be” can be taken out beyond the sum icon. By the way, formally this can be done with the sum 
Let's rewrite the system in “applied” form: 
after which the algorithm for solving our problem begins to emerge:
Do we know the coordinates of the points? We know. Amounts 
can we find it? Easily. Let's make the simplest system of two linear equations in two unknowns(“a” and “be”). We solve the system, for example, Cramer's method, as a result of which we obtain a stationary point. Checking sufficient condition for an extremum, we can verify that at this point the function 
reaches exactly minimum. The check involves additional calculations and therefore we will leave it behind the scenes (if necessary, the missing frame can be viewedHere
)
. We draw the final conclusion:
Function 
the best way (at least compared to any other linear function) brings experimental points closer . Roughly speaking, its graph passes as close as possible to these points. In tradition econometrics the resulting approximating function is also called paired linear regression equation
.
The problem under consideration is of great practical importance. In our example situation, Eq. allows you to predict what trade turnover ("Igrek") the store will have at one or another value of the sales area (one or another meaning of “x”). Yes, the resulting forecast will only be a forecast, but in many cases it will turn out to be quite accurate.
I will analyze just one problem with “real” numbers, since there are no difficulties in it - all calculations are at the level school curriculum 7-8 grades. In 95 percent of cases, you will be asked to find just a linear function, but at the very end of the article I will show that it is no more difficult to find the equations of the optimal hyperbola, exponential and some other functions.
In fact, all that remains is to distribute the promised goodies - so that you can learn to solve such examples not only accurately, but also quickly. We carefully study the standard:
Task
As a result of studying the relationship between two indicators, we obtained next pairs numbers: 
Using the least squares method, find the linear function that best approximates the empirical (experienced) data. Make a drawing on which to construct experimental points and a graph of the approximating function in a Cartesian rectangular coordinate system 
. Find the sum of squared deviations between the empirical and theoretical values. Find out if the feature would be better (from the point of view of the least squares method) bring experimental points closer.
Please note that the “x” meanings are natural, and this has a characteristic meaningful meaning, which I will talk about a little later; but they, of course, can also be fractional. In addition, depending on the content of a particular task, both “X” and “game” values can be completely or partially negative. Well, we have been given a “faceless” task, and we begin it solution:
We find the coefficients of the optimal function as a solution to the system: 
For the purpose of more compact recording, the “counter” variable can be omitted, since it is already clear that the summation is carried out from 1 to .
It is more convenient to calculate the required amounts in tabular form: 
Calculations can be carried out on a microcalculator, but it is much better to use Excel - both faster and without errors; watch a short video:
Thus, we get the following system:
Here you can multiply the second equation by 3 and subtract the 2nd from the 1st equation term by term. But this is luck - in practice, systems are often not a gift, and in such cases it saves Cramer's method:
, which means the system has a unique solution.
Let's check. I understand that you don’t want to, but why skip errors where they can absolutely not be missed? Let us substitute the found solution into left side each equation of the system:
The right-hand sides of the corresponding equations are obtained, which means that the system is solved correctly.
Thus, the desired approximating function: – from all linear functions It is she who best approximates the experimental data.
Unlike straight dependence of the store's turnover on its area, the found dependence is reverse (principle “the more, the less”), and this fact is immediately revealed by the negative slope. The function tells us that as a certain indicator increases by 1 unit, the value of the dependent indicator decreases average by 0.65 units. As they say, the higher the price of buckwheat, the less it is sold.
To plot the graph of the approximating function, we find its two values:
and execute the drawing: 
The constructed straight line is called trend line
(namely, a linear trend line, i.e. in the general case, a trend is not necessarily a straight line). Everyone is familiar with the expression “to be in trend,” and I think that this term does not need additional comments.
Let's calculate the sum of squared deviations 
between empirical and theoretical values. Geometrically, this is the sum of the squares of the lengths of the “raspberry” segments (two of which are so small that they are not even visible).
Let's summarize the calculations in a table: 
Again, they can be done manually; just in case, I’ll give an example for the 1st point: 
but it is much more effective to do it in the already known way:
We repeat once again: What is the meaning of the result obtained? From all linear functions the function has the smallest exponent, that is, it is the best approximation in its family. And here, by the way, the final question of the problem is not accidental: what if the proposed exponential function 
would it be better to bring the experimental points closer?
Let's find the corresponding sum of squared deviations - to distinguish, I will denote them by the letter “epsilon”. The technique is exactly the same:
And again, just in case, the calculations for the 1st point: 
In Excel we use the standard function EXP (syntax can be found in Excel Help).
Conclusion: , which means that the exponential function approximates the experimental points worse than the straight line.
But here it should be noted that “worse” is doesn't mean yet, what is wrong. Now I have built a graph of this exponential function - and it also passes close to the points - so much so that without analytical research it is difficult to say which function is more accurate.
This concludes the solution, and I return to the question of the natural values of the argument. In various studies, usually economic or sociological, natural “X’s” are used to number months, years or other equal time intervals. Consider, for example, the following problem:
The following data is available on the store’s retail turnover for the first half of the year:
Using analytical straight line alignment, determine the volume of turnover for July.
Yes, no problem: we number the months 1, 2, 3, 4, 5, 6 and use the usual algorithm, as a result of which we get an equation - the only thing is that when it comes to time, they usually use the letter “te” (although this is not critical). The resulting equation shows that in the first half of the year trade turnover increased by an average of 27.74 units. per month. Let's get the forecast for July (month no. 7): d.e.
And there are countless tasks like this. Those who wish can use an additional service, namely my Excel calculator (demo version), which solves the analyzed problem almost instantly! Working version of the program is available in exchange or for symbolic fee.
At the end of the lesson brief information o finding dependencies of some other types. Actually, there’s not much to tell, since the fundamental approach and solution algorithm remain the same.
Let us assume that the arrangement of the experimental points resembles a hyperbola. Then, to find the coefficients of the best hyperbola, you need to find the minimum of the function - anyone can carry out detailed calculations and arrive at a similar system: 
From a formal technical point of view, it is obtained from a “linear” system 
(let's denote it with an asterisk) replacing "x" with . Well, what about the amounts? 
calculate, after which to the optimal coefficients “a” and “be” close at hand.
If there is every reason to believe that the points are located along a logarithmic curve, then to find the optimal values we find the minimum of the function 
. Formally, in the system (*) needs to be replaced with: 
When performing calculations in Excel, use the function LN. I confess that it would not be particularly difficult for me to create calculators for each of the cases under consideration, but it would still be better if you “programmed” the calculations yourself. Lesson videos to help.
With exponential dependence the situation is a little more complicated. To reduce the matter to the linear case, we take the function logarithm and use properties of the logarithm:
Now, comparing the resulting function with the linear function, we come to the conclusion that in the system (*) must be replaced by , and – by . For convenience, let's denote: 
Please note that the system is resolved with respect to and, and therefore, after finding the roots, you must not forget to find the coefficient itself.
To bring experimental points closer optimal parabola 
, should be found minimum function of three variables 
. After performing standard actions, we get the following “working” system:
Yes, of course, there are more amounts here, but there are no difficulties at all when using your favorite application. And finally, I’ll tell you how to quickly perform a check using Excel and build the desired trend line: create a scatter plot, select any of the points with the mouse and right click select the option "Add trend line". Next, select the chart type and on the tab "Options" activate the option "Show equation on diagram". OK
As always, I would like to end the article with some in a beautiful phrase, and I almost typed “Be trendy!” But he changed his mind in time. And not because it is stereotyped. I don’t know how it is for anyone, but I don’t really want to follow the promoted American and especially European trend =) Therefore, I wish each of you to stick to your own line!
http://www.grandars.ru/student/vysshaya-matematika/metod-naimenshih-kvadratov.html
The least squares method is one of the most common and most developed due to its simplicity and efficiency of methods for estimating parameters of linear econometric models. At the same time, when using it, some caution should be observed, since models constructed using it may not satisfy a number of requirements for the quality of their parameters and, as a result, do not reflect the patterns of process development “well” enough.
Let us consider the procedure for estimating the parameters of a linear econometric model using the least squares method in more detail. Such a model in general can be represented by equation (1.2):
y t = a 0 + a 1 x 1t +...+ a n x nt + ε t.
The initial data when estimating the parameters a 0 , a 1 ,..., a n is a vector of values of the dependent variable y= (y 1 , y 2 , ... , y T)" and the matrix of values of independent variables
in which the first column, consisting of ones, corresponds to the model coefficient.
The least squares method received its name based on the basic principle that the parameter estimates obtained on its basis must satisfy: the sum of squares of the model error should be minimal.
Examples of solving problems using the least squares method
Example 2.1. The trading enterprise has a network of 12 stores, information on the activities of which is presented in table. 2.1.
The management of the enterprise would like to know how the size of the annual turnover depends on the retail space of the store.
Table 2.1
| Store number | Annual turnover, million rubles. | Retail area, thousand m2 |
| 19,76 | 0,24 | |
| 38,09 | 0,31 | |
| 40,95 | 0,55 | |
| 41,08 | 0,48 | |
| 56,29 | 0,78 | |
| 68,51 | 0,98 | |
| 75,01 | 0,94 | |
| 89,05 | 1,21 | |
| 91,13 | 1,29 | |
| 91,26 | 1,12 | |
| 99,84 | 1,29 | |
| 108,55 | 1,49 |
Least squares solution. Let us denote the annual turnover of the th store, million rubles; - retail area of the th store, thousand m2.
Fig.2.1. Scatterplot for Example 2.1
To determine the form of the functional relationship between the variables and we will construct a scatter diagram (Fig. 2.1).
Based on the scatter diagram, we can conclude that annual turnover is positively dependent on retail space (i.e., y will increase with increasing ). The most suitable form of functional connection is linear.
Information for further calculations is presented in table. 2.2. Using the least squares method, we estimate the parameters of a linear one-factor econometric model
Table 2.2
| t | y t | x 1t | y t 2 | x 1t 2 | x 1t y t |
| 19,76 | 0,24 | 390,4576 | 0,0576 | 4,7424 | |
| 38,09 | 0,31 | 1450,8481 | 0,0961 | 11,8079 | |
| 40,95 | 0,55 | 1676,9025 | 0,3025 | 22,5225 | |
| 41,08 | 0,48 | 1687,5664 | 0,2304 | 19,7184 | |
| 56,29 | 0,78 | 3168,5641 | 0,6084 | 43,9062 | |
| 68,51 | 0,98 | 4693,6201 | 0,9604 | 67,1398 | |
| 75,01 | 0,94 | 5626,5001 | 0,8836 | 70,5094 | |
| 89,05 | 1,21 | 7929,9025 | 1,4641 | 107,7505 | |
| 91,13 | 1,29 | 8304,6769 | 1,6641 | 117,5577 | |
| 91,26 | 1,12 | 8328,3876 | 1,2544 | 102,2112 | |
| 99,84 | 1,29 | 9968,0256 | 1,6641 | 128,7936 | |
| 108,55 | 1,49 | 11783,1025 | 2,2201 | 161,7395 | |
| S | 819,52 | 10,68 | 65008,554 | 11,4058 | 858,3991 |
| Average | 68,29 | 0,89 |
Thus,
Therefore, with an increase in retail space by 1 thousand m2, with other equal conditions average annual trade turnover increases by 67.8871 million rubles.
Example 2.2. The company's management noticed that the annual turnover depends not only on the store's sales area (see example 2.1), but also on the average number of visitors. The relevant information is presented in table. 2.3.
Table 2.3
Solution. Let us denote - the average number of visitors to the th store per day, thousand people.
To determine the form of the functional relationship between the variables and we will construct a scatter diagram (Fig. 2.2).
Based on the scatterplot, we can conclude that annual turnover is positively dependent on the average number of visitors per day (i.e., y will increase with increasing ). The form of functional dependence is linear.
Rice. 2.2. Scatterplot for Example 2.2
Table 2.4
| t | x 2t | x 2t 2 | y t x 2t | x 1t x 2t |
| 8,25 | 68,0625 | 163,02 | 1,98 | |
| 10,24 | 104,8575 | 390,0416 | 3,1744 | |
| 9,31 | 86,6761 | 381,2445 | 5,1205 | |
| 11,01 | 121,2201 | 452,2908 | 5,2848 | |
| 8,54 | 72,9316 | 480,7166 | 6,6612 | |
| 7,51 | 56,4001 | 514,5101 | 7,3598 | |
| 12,36 | 152,7696 | 927,1236 | 11,6184 | |
| 10,81 | 116,8561 | 962,6305 | 13,0801 | |
| 9,89 | 97,8121 | 901,2757 | 12,7581 | |
| 13,72 | 188,2384 | 1252,0872 | 15,3664 | |
| 12,27 | 150,5529 | 1225,0368 | 15,8283 | |
| 13,92 | 193,7664 | 1511,016 | 20,7408 | |
| S | 127,83 | 1410,44 | 9160,9934 | 118,9728 |
| Average | 10,65 |
In general, it is necessary to determine the parameters of a two-factor econometric model
y t = a 0 + a 1 x 1t + a 2 x 2t + ε t
The information required for further calculations is presented in table. 2.4.
Let us estimate the parameters of a linear two-factor econometric model using the least squares method.
Thus,
Estimation of the coefficient =61.6583 shows that, other things being equal, with an increase in retail space by 1 thousand m 2, the annual turnover will increase by an average of 61.6583 million rubles.
The coefficient estimate = 2.2748 shows that, other things being equal, with an increase in the average number of visitors per 1 thousand people. per day, annual turnover will increase by an average of 2.2748 million rubles.
Example 2.3. Using the information presented in table. 2.2 and 2.4, estimate the parameter of the one-factor econometric model
where is the centered value of the annual turnover of the th store, million rubles; - centered value of the average daily number of visitors to the t-th store, thousand people. (see examples 2.1-2.2).
Solution. Additional information required for calculations is presented in table. 2.5.
Table 2.5
| -48,53 | -2,40 | 5,7720 | 116,6013 | |
| -30,20 | -0,41 | 0,1702 | 12,4589 | |
| -27,34 | -1,34 | 1,8023 | 36,7084 | |
| -27,21 | 0,36 | 0,1278 | -9,7288 | |
| -12,00 | -2,11 | 4,4627 | 25,3570 | |
| 0,22 | -3,14 | 9,8753 | -0,6809 | |
| 6,72 | 1,71 | 2,9156 | 11,4687 | |
| 20,76 | 0,16 | 0,0348 | 3,2992 | |
| 22,84 | -0,76 | 0,5814 | -17,413 | |
| 22,97 | 3,07 | 9,4096 | 70,4503 | |
| 31,55 | 1,62 | 2,6163 | 51,0267 | |
| 40,26 | 3,27 | 10,6766 | 131,5387 | |
| Amount | 48,4344 | 431,0566 |
Using formula (2.35), we obtain
Thus,
http://www.cleverstudents.ru/articles/mnk.html
Example.
Experimental data on the values of variables X And at are given in the table. 
As a result of their alignment, the function is obtained 
Using least square method, approximate these data by a linear dependence y=ax+b(find parameters A And b). Find out which of the two lines better (in the sense of the least squares method) aligns the experimental data. Make a drawing.
Solution.
In our example n=5. We fill out the table for the convenience of calculating the amounts that are included in the formulas of the required coefficients. 
The values in the fourth row of the table are obtained by multiplying the values of the 2nd row by the values of the 3rd row for each number i.
The values in the fifth row of the table are obtained by squaring the values in the 2nd row for each number i.
The values in the last column of the table are the sums of the values across the rows.
We use the formulas of the least squares method to find the coefficients A And b. We substitute the corresponding values from the last column of the table into them: 
Hence, y = 0.165x+2.184- the desired approximating straight line.
It remains to find out which of the lines y = 0.165x+2.184 or better approximates the original data, that is, makes an estimate using the least squares method.
Proof.
So that when found A And b function took smallest value, it is necessary that at this point the matrix of the quadratic form of the second-order differential for the function 
was positive definite. Let's show it.
The second order differential has the form: 
That is
Therefore, the matrix of quadratic form has the form
and the values of the elements do not depend on A And b.
Let us show that the matrix is positive definite. To do this, the angular minors must be positive.
Angular minor of the first order 
. The inequality is strict, since the points
It is widely used in econometrics in the form of a clear economic interpretation of its parameters.
Linear regression comes down to finding an equation of the form
or 
Equation of the form 
allows based on specified parameter values X have theoretical values of the resultant characteristic, substituting the actual values of the factor into it X.
The construction of linear regression comes down to estimating its parameters - A And V. Linear regression parameter estimates can be found using different methods.
The classical approach to estimating linear regression parameters is based on least squares method(MNC).
The least squares method allows us to obtain such parameter estimates A And V, at which the sum of squared deviations of the actual values of the resultant characteristic (y) from calculated (theoretical) minimum:
To find the minimum of a function, you need to calculate the partial derivatives for each of the parameters A And b and set them equal to zero.
Let's denote 
through S, then:
Transforming the formula, we obtain the following system of normal equations for estimating parameters A And V:
Solving the system of normal equations (3.5) either by the method sequential elimination variables, or by the method of determinants, we find the required estimates of the parameters A And V.
Parameter V called the regression coefficient. Its value shows the average change in the result with a change in the factor by one unit.
The regression equation is always supplemented with an indicator of the closeness of the connection. When using linear regression, such an indicator is the linear correlation coefficient. There are different modifications of the linear correlation coefficient formula. Some of them are given below:
As is known, the linear correlation coefficient is within the limits: -1 ≤ ≤ 1.
To assess the quality of selection linear function square is calculated
Linear correlation coefficient called coefficient of determination. The coefficient of determination characterizes the proportion of variance of the resulting characteristic y, explained by regression, in the total variance of the resulting trait:
Accordingly, the value 1 characterizes the share of variance y, caused by the influence of other factors not taken into account in the model.
Questions for self-control
1. The essence of the least squares method?
2. How many variables does pairwise regression provide?
3. What coefficient determines the closeness of the connection between changes?
4. Within what limits is the coefficient of determination determined?
5. Estimation of parameter b in correlation-regression analysis?
1. Christopher Dougherty. Introduction to econometrics. - M.: INFRA - M, 2001 - 402 p.
2. S.A. Borodich. Econometrics. Minsk LLC “New Knowledge” 2001.
3. R.U. Rakhmetova Short course in econometrics. Tutorial. Almaty. 2004. -78p.
4. I.I. Eliseeva. Econometrics. - M.: “Finance and Statistics”, 2002
5. Monthly information and analytical magazine.
Nonlinear economic models. Nonlinear regression models. Transformation of variables.
Nonlinear economic models..
Transformation of variables.
Elasticity coefficient.
If there are nonlinear relationships between economic phenomena, then they are expressed using the corresponding nonlinear functions: for example, an equilateral hyperbola 
,
parabolas of the second degree 
and etc.
There are two classes of nonlinear regressions:
1. Regressions that are nonlinear with respect to the explanatory variables included in the analysis, but linear with respect to the estimated parameters, for example:
Polynomials of various degrees - 
, ;
Equilateral hyperbola - ;
Semilogarithmic function - .
2. Regressions that are nonlinear in the parameters being estimated, for example:
Power - ;
Demonstrative - ;
Exponential - .
The total sum of squared deviations of individual values of the resulting characteristic at from the average value is caused by the influence of many reasons. Let us conditionally divide the entire set of reasons into two groups: factor under study x And other factors.
If the factor does not influence the result, then the regression line on the graph is parallel to the axis Oh And 
Then the entire variance of the resulting characteristic is due to the influence of other factors and the total sum of squared deviations will coincide with the residual. If other factors do not influence the result, then y tied With X functionally and the residual sum of squares is zero. In this case, the sum of squared deviations explained by the regression is the same as the total sum of squares.
Since not all points of the correlation field lie on the regression line, their scatter always occurs as a result of the influence of the factor X, i.e. regression at By X, and caused by other causes (unexplained variation). The suitability of a regression line for forecasting depends on what part of the total variation of the trait at accounts for the explained variation
Obviously, if the sum of squared deviations due to regression is greater than the residual sum of squares, then the regression equation is statistically significant and the factor X has a significant impact on the result u.
, i.e., with the number of freedom of independent variation of a characteristic. The number of degrees of freedom is related to the number of units of the population n and the number of constants determined from it. In relation to the problem under study, the number of degrees of freedom should show how many independent deviations from P
The assessment of the significance of the regression equation as a whole is given using F-Fisher criterion. In this case, a null hypothesis is put forward that the regression coefficient is equal to zero, i.e. b = 0, and therefore the factor X does not affect the result u.
The immediate calculation of the F-test is preceded by analysis of variance. The central place in it is occupied by the decomposition of the total sum of squared deviations of a variable at from the average value at into two parts - “explained” and “unexplained”:
- total sum of squared deviations;
- sum of squared deviations explained by regression;
- residual sum of squared deviations.
Any sum of squared deviations is related to the number of degrees of freedom , i.e., with the number of freedom of independent variation of a characteristic. The number of degrees of freedom is related to the number of population units n and with the number of constants determined from it. In relation to the problem under study, the number of degrees of freedom should show how many independent deviations from P possible required to form a given sum of squares.
Dispersion per degree of freedomD.
F-ratios (F-test): 
If the null hypothesis is true, then the factor and residual variances do not differ from each other. For H 0, a refutation is necessary so that the factor dispersion exceeds the residual dispersion several times. The English statistician Snedekor developed tables of critical values F- relationships at different levels of significance of the null hypothesis and various numbers degrees of freedom. Table value F-criterion is the maximum value of the ratio of variances that can occur in case of random divergence for a given level of probability of the presence of the null hypothesis. Calculated value F-relationships are considered reliable if o is greater than the table.
In this case, the null hypothesis about the absence of a relationship between signs is rejected and a conclusion is drawn about the significance of this relationship: F fact > F table H 0 is rejected.
If the value is less than the tabulated F fact ‹, F table, then the probability of the null hypothesis is higher than a specified level and cannot be rejected without serious risk of drawing the wrong conclusion about the presence of a relationship. In this case, the regression equation is considered statistically insignificant. But he doesn’t deviate.
Standard error of regression coefficient
To assess the significance of the regression coefficient, its value is compared with its standard error, i.e. the actual value is determined t-Student's test: 
which is then compared with the table value at a certain significance level and number of degrees of freedom ( n- 2).
Standard parameter error A:
The significance of the linear correlation coefficient is checked based on the magnitude of the error correlation coefficient t r:
Total trait variance X: 
Multiple Linear Regression
Model building
Multiple regression represents a regression of an effective characteristic with two or more factors, i.e. a model of the form
Regression can give good results in modeling if the influence of other factors affecting the object of study can be neglected. The behavior of individual economic variables cannot be controlled, i.e. it is not possible to ensure the equality of all other conditions for assessing the influence of one factor under study. In this case, you should try to identify the influence of other factors by introducing them into the model, i.e., construct a multiple regression equation: y = a+b 1 x 1 +b 2 +…+b p x p + .
The main goal of multiple regression is to build a model with a large number of factors, while determining the influence of each of them separately, as well as their combined impact on the modeled indicator. The specification of the model includes two ranges of issues: selection of factors and choice of the type of regression equation
The task is to find the linear dependence coefficients at which the function of two variables A And b takes the smallest value. That is, given A And b the sum of squared deviations of the experimental data from the found straight line will be the smallest. This is the whole point of the least squares method.
Thus, solving the example comes down to finding the extremum of a function of two variables.
Deriving formulas for finding coefficients. A system of two equations with two unknowns is compiled and solved. Finding the partial derivatives of a function 
by variables A And b, we equate these derivatives to zero.
We solve the resulting system of equations using any method (for example, the substitution method or the Cramer method) and obtain formulas for finding the coefficients using the least squares method (LSM). 
Given A And b function 
takes the smallest value.
That's the whole method of least squares. Formula for finding the parameter a contains the sums , , , and parameter n- amount of experimental data. We recommend calculating the values of these amounts separately. Coefficient b found after calculation a.
The main area of application of such polynomials is the processing of experimental data (construction of empirical formulas). The fact is that an interpolation polynomial constructed from function values obtained through experiment will be strongly influenced by “experimental noise”; moreover, when interpolating, interpolation nodes cannot be repeated, i.e. The results of repeated experiments under the same conditions cannot be used. The root mean square polynomial smooths out noise and allows you to use the results of multiple experiments.
Numerical integration and differentiation. Example.
Numerical integration– calculation of the value of a definite integral (usually approximate). Numerical integration is understood as a set of numerical methods for finding the value of a certain integral.
Numerical differentiation– a set of methods for calculating the value of the derivative of a discretely specified function.
Integration
Formulation of the problem. Mathematical problem statement: you need to find the value definite integral
where a, b are finite, f(x) is continuous on [a, b].
When solving practical problems, it often happens that the integral is inconvenient or impossible to take analytically: it may not be expressed in elementary functions, the integrand can be specified in the form of a table, etc. In such cases, numerical integration methods are used. Numerical integration methods use replacing the area of a curvilinear trapezoid with a finite sum of areas of simpler ones geometric shapes, which can be calculated exactly. In this sense, they talk about using quadrature formulas.
Most methods use a representation of the integral as a finite sum (quadrature formula):
Quadrature formulas are based on the idea of replacing the graph of the integrand on the integration segment with functions of more simple type, which can be easily integrated analytically and thus easily calculated. The task of constructing quadrature formulas is most simply implemented for polynomial mathematical models.
Three groups of methods can be distinguished:
1. Method with dividing the integration segment into equal intervals. Partitioning into intervals is done in advance; usually the intervals are chosen equal (to make it easier to calculate the function at the ends of the intervals). Calculate areas and sum them up (rectangle, trapezoid, Simpson methods).
2. Methods with partitioning the integration segment using special points (Gauss method).
3. Calculation of integrals using random numbers (Monte Carlo method).
Rectangle method. Let the function (figure) need to be integrated numerically on the segment . Divide the segment by N equal intervals. The area of each of N curved trapezoids can be replaced by the area of a rectangle.
The width of all rectangles is the same and equals:
To select the height of the rectangles, you can select the value of the function on the left border. In this case, the height of the first rectangle will be f(a), the second - f(x 1),..., N-f(N-1).
If we take the value of the function on the right border to select the height of the rectangle, then in this case the height of the first rectangle will be f(x 1), the second - f(x 2), ..., N - f(x N).
As you can see, in this case one of the formulas gives an approximation to the integral with an excess, and the second with a deficiency. There is another way - to use the value of the function in the middle of the integration segment for approximation:
Estimation of the absolute error of the rectangle method (middle)
Estimation of the absolute error of the left and right rectangle methods.
Example. Calculate for the entire interval and dividing the interval into four sections
Solution. Analytical calculation of this integral gives I=arctg(1)–arctg(0)=0.7853981634. In our case:
1)h = 1; xo = 0; x1 = 1;
2) h = 0.25 (1/4); x0 = 0; x1 = 0.25; x2 = 0.5; x3 = 0.75; x4 = 1;
Let's calculate using the left rectangle method:
Let's calculate using the right rectangle method:
Let's calculate using the average rectangle method:
Trapezoid method. Using a first-degree polynomial (a straight line drawn through two points) to interpolate results in the trapezoidal formula. The ends of the integration segment are taken as interpolation nodes. Thus, curved trapezoid is replaced by a regular trapezoid, the area of which can be found as the product of half the sum of the bases and the height
In the case of N integration segments for all nodes, with the exception of the extreme points of the segment, the value of the function will be included in the total sum twice (since adjacent trapezoids have one common side)
The trapezoid formula can be obtained by taking half the sum of the formulas of rectangles along the right and left edges of the segment:
Checking the stability of the solution. As a rule, the shorter the length of each interval, i.e. how larger number these intervals, the less the difference between the approximate and exact value of the integral. This is true for most functions. In the trapezoid method, the error in calculating the integral ϭ is approximately proportional to the square of the integration step (ϭ ~ h 2). Thus, to calculate the integral of a certain function in terms of a, b, it is necessary to divide the segment into N 0 intervals and find the sum of the areas of the trapezoid. Then you need to increase the number of intervals N 1, again calculate the sum of the trapezoid and compare the resulting value with the previous result. This should be repeated until (N i) until the specified accuracy of the result is achieved (convergence criterion).
For the rectangle and trapezoid methods, usually at each iteration step the number of intervals increases by 2 times (N i +1 = 2N i).
Convergence criterion:
The main advantage of the trapezoidal rule is its simplicity. However, if high accuracy is required when calculating the integral, using this method may require too much large quantity iterations.
Absolute error of the trapezoidal method is estimated as
.
Example. Calculate an approximately definite integral using the trapezoidal formula.
a) Dividing the segment of integration into 3 parts.
b) Dividing the segment of integration into 5 parts.
Solution:
a) According to the condition, the integration segment must be divided into 3 parts, that is.
Let's calculate the length of each partition segment: 
.
Thus, general formula the trapezoid is reduced to a pleasant size:
Finally:
Let me remind you that the resulting value is an approximate value of the area.
b) Let's divide the integration segment into 5 equal parts, that is. By increasing the number of segments, we increase the accuracy of calculations.
If , then the trapezoidal formula takes next view:
Let's find the partition step: 
, that is, the length of each intermediate segment is 0.6.
When finalizing the task, it is convenient to formalize all calculations using a calculation table:
In the first line we write “counter”
As a result: 
Well, there really is a clarification, and a serious one!
If for 3 partition segments, then for 5 segments. If you take an even larger segment => it will be even more accurate.
Simpson's formula. The trapezoid formula gives a result that strongly depends on the step size h, which affects the accuracy of calculating a certain integral, especially in cases where the function is non-monotonic. It can be assumed that the accuracy of calculations will increase if, instead of straight segments replacing curvilinear fragments of the graph of the function f(x), we use, for example, fragments of parabolas given through three adjacent points of the graph. This geometric interpretation underlies Simpson's method for calculating the definite integral. Entire interval integration a,b N segments are divided, the length of the segment will also be equal to h=(b-a)/N.
Simpson's formula looks like:
remainder term
As the length of the segments increases, the accuracy of the formula decreases, so to increase the accuracy, Simpson's compound formula is used. The entire integration interval is divided into even number identical segments N, the length of the segment will also be equal to h=(b-a)/N. Simpson's compound formula is:
In the formula, the expressions in brackets represent the sums of the values of the integrand at the ends of the odd and even internal segments, respectively.
The remainder of Simpson's formula is proportional to the fourth power of the step:
Example: Using Simpson's rule, calculate the integral. (Exact solution - 0.2)
Gauss method
Gaussian quadrature formula. The basic principle of quadrature formulas of the second type is visible from Figure 1.12: it is necessary to place the points in this way X 0 and X 1 inside the segment [ a;b], so that the total area of the “triangles” is equal to the area of the “segment”. When using the Gauss formula, the original segment [ a;b] is reduced to the segment [-1;1] by replacing the variable X on
0.5∙(b– a)∙t+ 0.5∙(b + a).
Then 
, Where 
.
Such a replacement is possible if a And b are finite, and the function f(x) is continuous on [ a;b]. Gauss formula at n points x i, i=0,1,..,n-1 inside the segment [ a;b]:
, (1.27)
Where t i And A i for various n are given in reference books. For example, when n=2 
A 0 =A 1 =1; at n=3: t 0 =t 2 "0.775, t 1 =0, A 0 =A 2 "0.555, A 1 "0.889.
Gaussian quadrature formula
obtained with a weight function equal to unity p(x)= 1 and nodes x i, which are the roots of the Legendre polynomials
Odds A i easy to calculate using formulas
i=0,1,2,...n.
The values of nodes and coefficients for n=2,3,4,5 are given in the table
| Order | Nodes | Odds |
| n=2 | x 1=0 x 0 =-x 2=0.7745966692 | A 1=8/9 A 0 =A 2=5/9 |
| n=3 | x 2 =-x 1=0.3399810436 x 3 =-x 0=0.8611363116 | A 1 =A 2=0.6521451549 A 0 =A 3=0.6521451549 |
| n=4 | x 2 = 0 x 3 = -x 1 = 0.5384693101 x 4 =-x 0 =0.9061798459 | A 0 =0.568888899 A 3 =A 1 =0.4786286705 A 0 =A 4 =0.2869268851 |
| n=5 | x 5 = -x 0 =0.9324695142 x 4 = -x 1 =0.6612093865 x 3 = -x 2 =0.2386191861 | A 5 =A 0 =0.1713244924 A 4 =A 1 =0.3607615730 A 3 =A 2 =0.4679139346 |
Example. Calculate the value using the Gauss formula for n=2:
Exact value: 
.
The algorithm for calculating the integral using the Gauss formula does not involve doubling the number of microsegments, but increasing the number of ordinates by 1 and comparing the obtained values of the integral. The advantage of the Gauss formula is its high accuracy with a relatively small number of ordinates. Disadvantages: inconvenient for manual calculations; it is necessary to store the values in the computer memory t i, A i for various n.
The error of the Gaussian quadrature formula on the segment will be For the remainder term formula will be and the coefficient α N decreases quickly with growth N. Here 
Gaussian formulas provide high accuracy even with a small number of nodes (from 4 to 10). In this case, in practical calculations the number of nodes ranges from several hundred to several thousand. Note also that the weights of Gaussian quadratures are always positive, which ensures the stability of the algorithm for calculating the sums
Differentiation. When solving problems, it is often necessary to find the derivative of a certain order from the function f(x), given in a table. In addition, sometimes, due to the complexity of the analytical expression of the function f(x), its direct differentiation is too difficult, as well as when solving numerically differential equations. In these cases, numerical differentiation is used.
3.5. Least square method
The first work that laid the foundations of the least squares method was carried out by Legendre in 1805. In the article “New methods for determining the orbits of comets,” he wrote: “After all the conditions of the problem have been fully used, it is necessary to determine the coefficients so that the magnitude of their errors were the smallest possible. Most in a simple way to achieve this is a method that consists of finding the minimum sum of squared errors.” Currently, the method is used very widely when approximating unknown functional dependencies specified by many experimental samples in order to obtain an analytical expression that is best approximated to a full-scale experiment.
Let, on the basis of an experiment, it be necessary to establish the functional dependence of the quantity y from x : Let us assume that as a result of the experiment we obtainedn values yfor the corresponding values of the argumentx. If the experimental points are located on the coordinate plane as in the figure, then, knowing that errors occur during the experiment, we can assume that the dependence is linear, i.e.y= ax+ bNote that the method does not impose restrictions on the type of function, i.e. it can be applied to any functional dependencies.
From the experimenter's point of view, it is often more natural to consider that the sequence of samplingfixed in advance, i.e. is an independent variable, and counts - dependent variable. This is especially clear if under are understood as moments in time, which is most widely used in technical applications. But this is only a very common special case. For example, it is necessary to classify some samples by size. Then the independent variable will be the sample number, the dependent variable will be its individual size.
The least squares method is described in detail in many educational and scientific publications, especially in terms of approximation of functions in electrical and radio engineering, as well as in books on probability theory and mathematical statistics.
Let's return to the drawing. The dotted lines show that errors can arise not only due to imperfect measurement procedures, but also due to inaccuracy in specifying the independent variable. With the selected type of function 
All that remains is to select the parameters included in ita And bIt is clear that the number of parameters can be more than two, which is typical only for linear functions. In general, we will assume
.(1)
You need to select oddsa, b, c... so that the condition is fulfilled
.
(2)
Let's find the values a, b, c..., turning the left side of (2) to a minimum. To do this, we define stationary points(points at which the first derivative vanishes) by differentiating the left side of (2) with respect toa, b, c:
(3)
etc. The resulting system of equations contains as many equations as unknownsa, b, c…. It is impossible to solve such a system in a general form, so it is necessary to specify, at least approximately, a specific type of function. Next, we will consider two cases: linear and quadratic functions.
Linear function .
Consider the sum of squared differences experimental values and function values at the corresponding points:
(4)
Let's select the parametersa And bso that this amount has the smallest value. Thus, the task comes down to finding the valuesa And b, at which the function has a minimum, i.e. to study the function of two independent variablesa And bto a minimum. To do this, we differentiate bya And b:
;
.
Or
(5)
Substituting the experimental data and , we obtain a system of two linear equations with two unknownsa And b. Having solved this system, we can write the function .
Let us make sure that for the found valuesa And bhas a minimum. To do this, we find , and :
, 
, .
Hence,
− = 
,
>0,
those. a sufficient minimum condition for a function of two variables is satisfied.
Quadratic function 
.
Let the experiment obtain the values of the function at points . Let also, based on a priori information, there be an assumption that the function is quadratic:
.
We need to find the coefficientsa, b And c.We have
– function of three variablesa,
b,
c.
In this case, system (3) takes the form:
Or:
Having solved this system of linear equations, we determine the unknownsa, b, c.
Example.Let four values of the desired function be obtained based on the experiment y = (x ) with four values of the argument, which are given in the table: