@Carl Love My reply to the OP was in reaction to his statement that he is finding three local minima and wishes to pick the best among them.  That doesn't inspire me with confidence that he understands the basics of the problem, hence my suggestion.

Forget about Maple for now. Try finding the minima by hand.  You will find that illuminating.

I can't understand at all what you are asking, and apparently neither does anyone else.

It will help if you could give one or two examples, consisting of:what you wish to type into Maple, and the result that you wish Maple to produce.

For example, you may say:

I want to type sin(2*x) into Maple and I want Maple to produce 2*sin(x)*cos(x).

Why would you expect a symbolic solution for this system?  You have two things going against you: (A) it is nonlinear; (B) is has variable coefficients.

You should be able to calculate a solution numerically if you specify the coefficients eta(t), gama(t), etc., and initial conditions.

Expanding on Kitonum's suggestion:

restart;
macro(D=`&D;`, I=`&I;`);
I = (1/64)*(D^4-d^4)*Pi;

@WillG The system, as you have posed, has no solution in general. Read the paper that you are referring to carefully to see what other assumptions are being made.

@tomleslie 

The conditions that I have imposed are not entirely arbitrary.  The PDE under consideration is a linearized version of the well-studied Korteweg de Vries (KdV) equations (look up KdV in Wikipedia).  It models dispersive waves on the entire real line (x going from minus infinity to plus infinity).  The solutions of interest are dispersive waves that propagate along the x axis while the far regions of the x axis are in undisturbed states, that is, the limits of the solution at plus/minus infinity are zero. Note that the OP specifically refered to dispersive waves.

For numerical calculations we replace infinity with finite values and move the "undisturbed" conditions to interval's boundaries.  The solution thus obtained is good for smallish values of time. When the wave interaction with the (artificial) boundaries becomes significant, the solution ceases to be meaningful.  To get good solutions for larger values of time, we solve the PDE over a larger x interval.  In the code I presented, I left the coordinates of the end points, a and b, as parameters in order to facilitate numerical experiments.

@ijuptilk That goes beyond my areas of interest.  I will let someone else to handle it.

@ijuptilk Here is the implementation of that modified algorithm in Maple.  Its results agree with what you have shown.

Download mw2.mw

Running out of memory can cause those symptoms.  Perhaps you are running several other memory-consuming applications at the same time?  Try closing your browser and other memory hogs.

@Tokoro That's a good question.  I hadn't thought about it.

My calculations show that if we split the disk into three sectors of arbitrary sizes, then the maximum total volume is achieved when one of the sectors is of zero angle, and thus the problem reduces to the two cones case.

On the other hand, if we split the disk into three sectors so that two of the sectors are of the same size, then the maximum total volume is achieved when one of the sectors is of zero angle, and the other two are exactly half of the disk each.

@FredK You may have any number of equations and functions within a single proc.  It's difficult to provide a more concrete answer in the absence of more detailed information.

The expression
    - x(t) - 1/6*x(t)^3 + 1/120*x(t)^5
looks very much like the  first few terms of the series expansion of the sine function but the signs are wrong.  I am guessing that it is meant to be
    - x(t) + 1/6*x(t)^3 - 1/120*x(t)^5

Same goes for the y(t).

How would you solve this problem with paper and pencil?  If you present an outline that shows that you understand the math, then I will be happy to show you how to code that math into Maple.

There are very many ways of doing that, some more sophisticated that others, and it's not all about knowing how to use Maple; tt also depends on what sort of mathematics you know.

Can you describe how you would construct the triangle if you were doing this a hundred years ago, before there were computers? Your (detailed) answer to that question will help us pick and present to you the solution that.best matches your background.