Understanding CAL data

This article is a grab-bag of hints for CAL programmers. Its emphasis is on CAL data types and how they are used.

CAL variable names

Case sensitivity: The CAL interpreter is case sensitive. (This is covered in the documentation, but it is important enough to repeat.) pause is a Cakewalk reserved word. PAUSE, Pause and PaUse are interpreted as different variables.

Any printable characters: CAL lets you use any combination of printable characters as a variable, except for CAL reserved words. Still, if you declare a number as a variable, some very interesting things could happen. Let me know what happens if you try this! Variable names that start with a digit do work OK as long as they have at least one non-numeric character, but they are best avoided. If (for example) you use 2^16 as a variable but you forget to declare it, CAL will evaluate it as 2, since a string beginning with a number is evaluated until the first non-numeric character. Avoid this confusion by using Hungarian notation, like (dword d2^16 65536), which also shows the data type.

Use Hungarian notation! Variables in Hungarian notation, such as dDoubleLengthDataWord and nIntegerVariable makes it easy to keep track of each variable's data type. Long, descriptive variable names are a big help when debugging.

Only the variable types shown in boldface are actual CAL data types. A time must be declared as a long or a dword. If you declare it as a word or an int, your CAL program will bomb the first time it encounters a long piece of music. A Boolean may be declared as any of the numeric types and is commonly used in a conditional. Statements inside the parentheses of (if bTrue ...) will execute if the value of bTrue is anything other than 0.

Data entry

 CAL's getInt and getWord data entry commands allow you to set minimum and maximum values for data entry. You can do this with a getTime command by enclosing it in a while loop. A warning message appears if the entry is outside the allowed range. Either getInt, getWord or getTime can write data to any type of CAL numeric variable (not just the one that corresponds to their data limits). The data monitoring with getInt is better than with getWord, so use getInt whenever possible (that is, whenever the value to enter will always be less than 32767). Some problems to watch out for:

• The getWord command allows keyboard entry of non-numeric characters, even though it only outputs numeric data. CAL stops reading a getWord entry with the first non-numeric character, and issues no warning. For example, you can enter 23e5 and CAL will read 23. Use getInt except when you need the larger values of positive integers that getWord allows.
• getWord and GetInt let you enter a number which overflows the input data word. Then the data entry value "rolls over." To give an extreme example, if you set the min and max limits in of getWord to 0 and 65535, no data checking is performed at all. Then, if you enter 65536, getWord returns 0. If you enter 65537, getWord returns 1 -- and so forth. 
• The data checking parameters also roll around. For example, entering a min and max of 1 -1 in getWord is the same as entering 1 65535.

To avoid wildly unexpected results, I recommend echoing or testing input data following a getInt,. getWord or getTime command. You can set up sophisticated input testing with a while loop. You can also use variables for the minimum and maximum alllowed values.

Data limits

The limits for CAL data types are documented in the help screens and the table (above) in this article, but some important limits are not. It is much easier to exceed limits with a CAL program than with onscreen editing. Here are some additional limits determined by experiment:

File time limit in recent Cakewalk versions is 16,777,215 (2^24 - 1) ticks. This is a rather high limit -- but on the other hand, most Cakewalk commands provide no error trapping for overruns; the data just rolls over through zero.

What does that mean? Here's an example. Tick 16,777,215 falls at measure 34593:3:015 in 4/4 time at 120 ticks per quarter note. Try entering an event 3 or 50,or 500 measures later, in Event list view: when you hit the return key, the event will be written in measure 3, or 50, or 500. Clearly, you don't want your CAL programs writing events past the limit -- they will appear instead at unexpected places below the limit.

Be aware that not all Cakewalk and CAL commands may able to work at or near the limit. I have found that the Retrograde command, for example, gives strange results with high time values.

In Cakewalk 8, it is possible to go to time values above 16,777,215 ticks using a comand such as (= Now 33445432). The Now line will indicate these times with no problem in Track View -- but it is not possible to place events at times beyond the limit. And Piano Roll in Cakewalk 8 gets weird at approximately 2^23 ticks -- all the measure and beat lines disappear, and notes do not display. This behavior in Piano Roll does not seem to affect the ability to store and retrieve events in CAL.

meas, beat and tick: meas correctly calculates measure numbers for measures beyond the Cakewalk limit of 16,777,215 ticks. It is therefore psosible to trap measure numbers which would cause event times to roll around.

Largest and smallest number of ticks per second:

Largest and smallest number of ticks per measure:

The largest number of ticks per measure in Staff View results from the limitation in measure length in Staff View to 16 quarter notes or equivalent. The limit is reached at meters of 64/16, 32/8, 16/4, 8/2 and 4/1 (or larger numerators with the same denominators), none of which is common.

Real time limit depends on the highest tick number, TIMEBASE and tempo:

Even at the highest tempo and TIMEBASE settings, Cakewalk can accommodate a work well over two hours long. At more usual tempo and timebase settings, Cakewalk will run for nearly 24 hours. These calculations are not only a mental exercise: times of several hours are common and practical in theatrical MIDI applications.

By setting the timebase and tempo as low as possible, you could use Cakewalk as a timer that would run for over four weeks, assuming that the computer or Windows doesn't crash first!

Long times can also be accommodated by playing back a series of files using Cakewalk's Virtual Jukebox or play list. While editing, it is easier to keep all material in one file in Cakewalk versions up through 6. Versions 7 and up let you open more than one document at once, and so in these versions the data is easier to access if divided up into smaller files. Separate files make more sense when storing music for playback.

Times greater than the limit will roll around through zero -- a time of 16,777,216 will be recorded as zero, 16,777,217 as 1, etc.

Note length limit is 65535 (2^16 - 1) ticks. This is 546:015 quarter notes at 120 ticks per quarter. The note length limit corresponding to the timebase and meters in use in your file will appear in the upper right corner of the Event Filter dialog box.

Least common multiplier of all Cakewalk timebases: 40320, which is 7 * 5 * 3² * 2⁷. This is useful in calculations, so that they will always come out in an integral number of ticks.

Largest number of options in a switch expression: 122 (In CPA 7. Your mileage may vary.) It is easy to split up a switch expression into two or more shorter ones.

The EditSlide and EditSlide40 commands in Cakewalk versions through 6.01 take a minimum argument of -32768 (- 2^15) and a maximum argument of 32767 (2^15 - 1) regardless of the units, be they ticks, measures, seconds or frames. This limitation may force you to carry out a slide in two steps, one for measures and the other for ticks. The error message you get when trying to slide too far is "value out of range, xx" where xx is the number or variable name of the argument. This message can be confusing, because the value isn't necessarily out of range, rather, the command is out of its range. The EditSlide40 command in version 7.0 and higher takes a long integer as an argument. There's more about the Slide commands in another file on this site.

BUG warning! The calculation of frames in the EditSlide40 command under CPA 6.x is inaccurate if you use tempo changes in your music! Count by seconds and then by frames to avoid errors which can be as high as several minutes in an hour. This problem seems to have been fixed in CPA 7.0 and later, but check to be sure.

CAL file size limit documented in the CPW 3.0 manual is 32768 characters. It is possible to extend this limit by using included files, as long as the limit is not exceeded in any one use of the program. It may even be possible to exceed the limit in one use, though I haven't tested that yet. The CAL view, at least in v. 7, will sometimes stop entering characters if the file becomes larger than approximately 22 kB.

Nesting depth for CAL expressions appears to be at least 17. That is, there can be 17 sets of parentheses one inside the next. I tested this in CPA 6.01. Nesting depth may be different in other versions or with different expressions from those I used. The file I used to test nesting depth is on this site.

 CAL arithmetic

The LISP-like syntax of CAL arithmetic can at first be confusing, since it isn't the same as standard arithmetic, and not even the same as the reverse Polish arithmetic of old Hewlett-Packard calculators (Hungarian, Polish -- what is this thing about Eastern Europe, anyway?)

Still, you will soon get used to CAL arithmetic. But be careful about the order of operands. The order does not matter in

 The order does matter in

It helps to think of CAL as moving the operator ahead of the first operand.. So, when you would normally think (10 / 2), change this to (/ 10 2) for CAL.

And remember that CAL performs integer division. (/3 2) is 1 in CAL. (%3 2) gives you the remainder, also 1 in this case. To minimize roundoff errors, perform multiplication before dividing or findng the remainder. For example, to multiply 7 by 5/4:

(/ 4 (* 7 5)) gives the correct integer result, 8, and

(% 4 (* 7 5)) gives the correct remainder, 3.

But if we divide first,

(* 5 (/ 7 4)) gives an incorrect integer result, 5

(because 7 /4 is 1.75, whose integer part is 1); and similarly,

(% 5 (/ 7 4)) gives an incorrect integer result, 1.

Division with a negative result and a remainder always gives the nest higher rather than next lower integer, and gives a negative remainder if the variable receiving the remainder can take on negative values.:

(/ 7 4) gives the result 1.

(/ -7 4) gives the result -1.

Therefore, a "stairstep" function calculated by division and which passes through zero will have a double-width step at zero:

(/ 3 4) gives the result 0.

(/ -3 4) also gives the result 0.

This problem can be overcome by adding 1 to any negative result when there is a remainder. Beware of storing a negative result of a division, or a remainder of a division whose result is negative, in a word or dword variable.

gives the result 65533. Not right.

gives the result 65535.

Positive quotients may be rounded to the nearest integer by adding half the divisor to the dividend:

(= wRoundedQuotient (/ (+ wDividend (/ wDivisor 2)) wDivisor))

Automatic data type conversions

Dyed-in-the-wool C programmers may skip the following section. You know all this stuff already. The rest of you, try the CAL programs in this section. They are a good way to get familiar with the CAL View, and they give some interesting results. Consider this program fragment:

The result should set b equal to a, right? No -- the result will be 2, because the word data type has an upper limit of 65535. As with the getWord command, the value of the result rolls over if the data type does not support it. Think of the numbers a data type can store as being in a circle. As you go around the circle in the word data type, the numbers get as high as 65535, but the next number is zero, and then you start all over again.

Here's another example, probably even more common and confusing:

The result looks as though it should be 65536, but it is 0. When two variables of type word are multiplied, the result is stored as a variable of type word, and the result rolls over. On the other hand,

gives the correct result of 65536. When a word (maximum of 65535) and a dword (maximum of 4,294,967,295) are multiplied together, the result is stored as a dword. Similar rules apply to the other data types: the data is "promoted" to the higher data type. When in doubt, write a short program like the ones I've just given and check out the results.

Now, run this program and see if the results are what you expect:

The result should be -1, right?

The result is slightly inaccurate, 4,294, 967,295. That's because the dword variable type can only hold zero and positive numbers. Ask it to hold a number one less than zero, and it instead will roll around to the very largest number it can hold.

In this example, a data type change to long, which can hold negative numbers, does not occur, even though one of the two operands is a long. On the other hand,

gives the correct result, -1. The result is stored into the variable type that comes first in the expression. But be careful!

looks as though its result should be -16129, but the result is 49407, or 65536 - 16129. The word data type can not hold a negative number.

In elementary school, we learn that two numbers to be multiplied can be put in each other's place without changing the result. That's true enough. What can NOT be exchanged in this example is the data types -- simple data promotion is not possible, because the word and int data types have overlapping ranges. int takes -32768 to 32767, and word takes 0 to 65535. Neither type fully includes the range of the other. In developing the CAL language (as with other computer languages), the data type of the result had to depend on the position of the variables in the expression. When data ranges overlap, the data type which comes first becomes the data type of the result.

Well, almost always. The following program gives an even more incorrect result, 8306943 in Cakewalk 7.0 (though it gives -16129 in Cakewalk 8.0):

The int type could hold the correct result, but the value I tried to store in the word variable is impossible. When I first posted this article, I asked if any reader could figure out how CAL got this result.. Tom Kish, r36971@email.sps.mot.com, figured it out:

Thanks, Tom!

The following program,

gives the correct result, because both variables hold values which are possible for them, and because the result is stored to the int data type, which can correctly hold the result.

Automatic data type conversions can be confusing. If you are in any doubt, don't be ashamed to convert the data into a type you are sure will work before you perform arithmetic using it. You have been warned!

 The selection in CAL

CAL has passed through several versions. The most important difference between recent versions is in how they handle the selection.

Versions through 3.0 were bundled with Cakewalk products which used the Track/Measure view (a measure grid with dots -- no "clips"). These Cakewalk products could only make a selection by time and track, or by using the event filter.

More recent Cakewalk products allow independent selection of each event. While the selection options in these new Cakewalk products are nice, CAL hasn't entirely kept up with them.

CAL can only create a selection in the old way, by track and time or by using the Event Filter. CAL still can not mark individual events for selection.. The most unfortunate consequence is that CAL can not restore a complicated selection after some editing function has required a different selection. You often can get around this problem by placing editing functions that require a broad selection first, and then narrowing the selection for use with other functions. A more powerful technique is to slide the selection past the End time, work on it there, and then slide it back.

The FileExtract and FileMerge commands would provide a way around this limitation, except that the FileExtract command deletes empty tracks until it finds the first track containing data. For this reason, the track numbers may change when you paste the data back in. One way to get around this problem is to make track 1 a dummy track which you archive so it never sounds, and to include it in every selection which you are going to extract.

Some CAL limitations

CAL can manipulate MIDI data many ways, but:

While CAL can duplicate the functions of many Cakewalk dialog boxes, there is no way to open up a Cakewalk dialog box from inside CAL. You are limited to numeric data entry, one number at a time unless (perhaps -- my knowledge does not extend that far) you use CAL's DLL call.

Only one Insert Menu command, Insert Series of Controllers, is partly supported in CAL. It is documented in the CAL Edit Menu help screen. The article about CAL Edit Menu commands on this site gives fuller documentation.

CAL can insert events which it supports as data types, and it can insert other types of data by using the FileMerge command. This command still words in newer versions of CAL in which it has been dropped from the help screens. There is a complete list of CAL data types and how they are supported in another article on this site.

The following commands from the Insert menu are supported in CAL only through CAL's calling Cakewalk menu commands, because CAL does not recognize the data types on which these commands work:

Tempo Change
Meter/key change
Series of Tempos 
Marker

(However, programming tricks allow CAL to read all of these types of events except for markers, and CAL can insert them all using the FileMerge command.)

Nothing from the Realtime menu is available in CAL, because CAL doesn't run in real time, while the music is playing or recording.

CAL can write, but not not read, track parameters. You can usually get around this limitation by selecting one track at a time to perform operations on the data it contains. What you can't do is select some of the data in each of several tracks and then look at that data one track at a time -- unless you use the trick of moving all of that data past the end of the file, working on it, and then moving it back.

All of the track properties and initial track settings such as Pan, Volume, Patch and Port can be set in CAL, but they can not be read in CAL. You can put controller events into the track to change initial track settings which correspond to controller numbers, but you can not use CAL to automate operations which depend on track parameters. In my opinion, this is CAL's worst shortcoming. CAL appears to have been created in the days when multi-synth studios were rare. The inability to perform different actions depending on port assignment is very limiting now that such studios are common. In the Track menu, Solo/Unsolo, Kill, Wipe and Sort also do not work. You can effectively wipe a track by selecting it and then deleting all its events.

 The Virtual Piano and Virtual Jukebox (in the Tools menu) do not work in CAL. These are real-time commands.

In the GoTo Menu, the Previous Measure and Next Measure commands do not work in CAL. This is unimportant, because you can achieve the same results using other commands.