manipulate - Clicking to add/remove a Locator

I would like to add a Locator to a figure displayed by Manipulate when I click inside the figure, at the location where I click, and to have this trigger an update of the figure. I would also like to have clicking on an existing Locator remove it, and dragging a locator reposition it (again, redrawing the figure in response in each case).

For (a silly) example, I would like clicking anywhere in the region displayed by

With[{locs = {}}, 
 Manipulate[Graphics[Text[locs, {0, 0}], PlotRange -> 2],
  {{q, {1, 1}}, Locator, 
   Appearance -> Graphics[{Green, Disk[{0, 0}, .1]}, ImageSize -> 10]},

  {{p, locs}, Locator, 
   Appearance -> Graphics[{Red, Disk[{0, 0}, .1]}, ImageSize -> 10]}]]

to add the corresponding Point to locs, and for clicking on any p Locator to remove it from locs, with the text, updating accordingly in response to additions, deletions or repositionings of a Locator.

To make things hard, I have another locator q that needs to coexist with the one to which I would like to add/remove; and p and q have different appearances.

Answer

I would approach the overall goal of the program in a different way that avoids the limitations* of Manipulate. I would store a list of all the locators in one variable, with the permanent locator(s) at the beginning of the list. The transient locators can be added and removed with ALT+click. The permanent locators are maintained by the option of the form LocatorAutoCreate -> min, where min is the number of permanent locators.

Using With to assign symbols to represent the parts of the list locs of locators seems slightly inelegant, but overall the approach has the advantage of being simple.

Manipulate[
 With[{q = locs[[1]], p = Drop[locs, 1]}, 

  Graphics[Text[locs, {0, 0}], PlotRange -> 2]
  ],
 {{locs, {{1, 1}}}, Locator, LocatorAutoCreate -> 1}
 ]

---

*Limitation: When several Locator controls, p, q, ..., appear in a Manipulate, a LocatorPane is created of the form

LocatorPane[Dynamic[{p, q, ...}], ...]

The option LocatorAutoCreate can be passed to the LocatorPane; however, it does not work because the length of the list {p, q, ...} cannot be changed.

---

Update: Appearance

The appearance of the locators may be controlled in two ways. The documented, and hence common, solution is to use invisible locators and insert graphics primitives into the Graphics:

Manipulate[
 With[{q = locs[[1]], p = Drop[locs, 1]}, 
  Graphics[{
    Text[locs, {0, 0}],
    Locator[q], {Red, PointSize[Large], Point[p]},
   PlotRange -> 2]
  ],

 {{locs, {{1, 1}}}, Locator, LocatorAutoCreate -> 1}
 ]

Another way is to use the option setting

Appearance -> {Automatic, Graphics[{Red, Disk[]}, ImageSize -> 10]}

The last specification is repeated when there are more than two locators, a behavior that I could not find documented.