# To avoid the difficulty of the nonsmooth constraints

To avoid the difficulty of the nonsmooth constraints, a sequence of smooth problems by using smoothing functions have been progressively approximated this nonsmooth problem. To date, there have been many smoothing functions, such as perturbed Fischer–Burmeister (FB), Chen–Harker–Kanzow–Smale (CHKS), Neural networks, uniform, and Picard smoothing functions [18], [19], [20]. Thereinto, perturbed FB and CHKS smoothing functions have been applied widely to solve BLP problems. Facchinei et al. [21] transformed mathematical program with equilibrium constraints (MPEC), including bilevel programming problems as a particular case, into an equivalent one-level nonsmooth optimization problem, which was then approximated progressively by applying CHKS smoothing function to a sequence of smooth, regular problems that approximate the nonsmooth problem. Finally, it Refametinib was solved by standard available software for constrained optimization. Dempe [22] introduced a smoothing technique by applying perturbed FB function and Lagrangian function method to solve a BLP problem, approximated by a sequence of smooth optimization problems. Wang et al. [23] proposed an approximate programming method based on the simplex method to research the general bilevel nonlinear programming model by applying perturbed FB smoothing function. Etoa [24] presented a smoothing sequential quadratic programming using perturbed FB functional to compute a solution of a quadratic convex bilevel programming problem. In general, perturbed FB and CHSK smoothing functions are very closely related. These two functions are very similar except for the difference of a constant. In what follows, we choose to focus on the CHKS smoothing function to approximate nonsmooth problem.
The purpose of this work is to design an efficient algorithm for solving general nonlinear bilevel programming (NBLP) models. Different from the algorithms in [22], [23], where Lagrangian optimization and simplex method were used to solve a smooth problem approximated by perturbed FB smoothing function, we apply a CHKS smoothing function and an augmented Lagrangian multiplier method to solve NBLP problem, and focus on the detailed analysis on the condition for solution optimality. The remainder of this paper is organized as follows: Section 2 presents the formulation and basic definitions of bilevel programming, while Section 3 introduces the smoothing method for nonlinear complementarity problems. In Section 4 we explain the augmented Lagrangian multiplier method in detail and derive the condition for asymptotic stability, solution feasibility, and solution optimality. Numerical examples are reported in Section 5 and Section 6 concludes the paper.

Formulation of an NBLP problem
The general formulation of NBLP problems is as follows [25]:where represent the objective function of the leader and the follower, respectively, denotes the constraints of the leader, denotes the constraints of the follower, and denote the decision variables of the leader and the follower, respectively. The following definitions are associated with the problem (1). The relaxed feasible set (or constrained region) is defined as S= (x, y): G(x, y)⩽0, g(x, y)⩽0 , the lower level feasible set for each fixed x as S(x)= y: g(x, y)⩽0 , the follower’s rational reaction set as , and the inducible region as IR= (x, y): (x, y)∈S, y∈P(x) .
To guarantee that there is at least one solution to the NBLP problem, we assume that the constrained region S is nonempty and compact. At the same time, for some fixed x∈X, we assume that the lower level problem is a convex optimization problem satisfying the Mangasarian–Fromovitz constraints qualification (MFCQ) for any y∈S(x). Then we can reduce the bilevel programming problem to a one-level problem using the KKT condition.where is a Lagrange function.

Smoothing method
Problem (2) is a mathematical program with nonlinear complementary constraints. It is non-convex, non-differentiable and the regularity assumptions, which are needed to handle smooth optimization programs successfully, are never satisfied. So tube nucleus is not appropriate to use standard nonlinear programming software to solve it [26], [27].