Announcement

Collapse
No announcement yet.

Initial Python reference

Collapse
This is a sticky topic.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Initial Python reference

    General
    This will just be the start, much more is to follow, but a combination of having to get the scenarios done in time and RL issues made me fall behind schedule a bit.

    There are basically 4 ways to use Python:

    1) Map generation - there won't be additional documentation for this in the short run, but the existing scripts are well documented so you can learn a lot by studying those. Creating completely new map scripts is not for the faint-of-heart though, some grasp of OO programming and the mathemetical concepts underlying the scripts is required. You can find the mapscripts in the PublicMaps folder
    2) UI modding - documentation here isn't really needed, the code is more or less self-explanatory. You can find the UI files in the Assets/Python/Screens folder
    3) Events - the most common way in which you will use Python, there are many different applications, not just for your standard scenario events. A list of available events is below. The core event manager is Assets/Python/CvEventManager.py
    4) Overrides - some game functions can be overwritten, for example what can be built in what cities, what civics/techs/religions/etc are available to who (all dynamically, so on a case-by-case and turn-by-turn basis, much more powerful than the static XML files), some AI behaviour, Barbarian spawning, victory conditions, etc. These overrides are in Assets/Python/CvGameUtils.py

    There are some other very specific Python applications, but those you'll normally not need. In the short run at least, there will be no documentation for them (just give me and the community time though )

    A note of caution: please do NOT, I repeat, do NOT mess with the Python files in the default Assets folder, this will cause all kinds of problems with future patching, MP play, tournaments, etc. Please use the mod setup that's also used for the in-game scenarios, see the Mods folder. Here you only need to keep the things you add/change, files that remain unchanged you can leave out, the game will just take the version from the default Assets folder for any file it's missing (similar to CtP modding, for those familiar with it -- only this time it actually works). More detailed info will follow but I think most people will be able to figure this out soon enough, it's pretty simple.

    API
    As any programmer knows, to help you write Python code that actually interacts with the Civ4 engine, rather than just being a stand-alone application, you will need to use a so-called API (Application Programming Interface) -- basically a list of functions you can use to access routines in the game engine. I made a script to extract this info in HTML form from the SDK, but of course you guys don't have the SDK yet, so I'll just have to publish the HTML files. A few isues to keep in mind:

    1) My script can only get info from the SDK, while some of the API calls are not in the SDK (game-engine related stuff). I had to add those functions manually and left them mostly undocumented (for now). Classes that have a lot of bright green ??? in them are basically the ones that are not from the SDK.
    2) Documentation on what these functions do is pretty poor, I'm working on more complete info. Most of the time the name of the function pretty much tells you what it does though, so this shouldn't keep anyone from using them (it certainly hasn't kept me).
    3) This is for a not-quite-final version of the game, there might be some minor changes in the release version but it should be only really minor stuff, if anything changed at all. For an even-closer-but-still-not-quite-final build my script is FUBARed, I'll have to check what goes wrong there, and then ask Soren or Trip to run my script on the release version of the SDK (which I don't have myself either) to get this fully up-to-date (there are no changes to the API itself between these two builds though). For now this version will work absolutely fine though, at least 99.99% of this should still be accurate.
    4) Hmm, there was another issue but I forgot. Oh well, I'll update this post when I remember it.

    You can browse the API reference here: http://civilization4.net/files/modding/PythonAPI/
    A zip version that you can extract and browse on your own PC is here: http://civilization4.net/files/modding/PythonAPI.zip

    Events
    As promised, the list of events that's available to you in the game (sorry for the poor formatting, I don't have terribly much time today). This was also generated based on source files (with a few notes from myself) for a not-quite-final build, more detailed and final info will follow:

    onKbdEvent
    'keypress handler - return 1 if the event was consumed'
    arguments: eventType,key,mx,my,px,py

    onInit
    'Called when Civ starts up'
    arguments: -

    onUpdate
    'Called every frame'
    arguments: fDeltaTime

    onWindowActivation
    'Called when the game window activates or deactivates'
    arguements: bActive

    onUnInit
    'Called when Civ shuts down'
    arguments: -

    onPreSave
    "called before a game is actually saved"
    arguments: -

    onSaveGame
    "return the string to be saved - Must be a string"
    arguments: -

    onLoadGame
    -
    arguments: -

    onGameStart
    'Called at the start of the game'
    arguments: -

    onGameEnd
    'Called at the End of the game'
    arguments: -

    onBeginGameTurn
    'Called at the beginning of the end of each turn'
    arguments: iGameTurn

    onEndGameTurn
    'Called at the end of the end of each turn'
    arguments: iGameTurn

    onBeginPlayerTurn
    'Called at the beginning of a players turn'
    arguments: iGameTurn, iPlayer

    onEndPlayerTurn
    'Called at the end of a players turn'
    arguments: iGameTurn, iPlayer

    onEndTurnReady
    -
    arguments: iGameTurn

    onFirstContact
    'Contact'
    arguments: iTeamX,iHasMetTeamY

    onCombatResult
    'Combat Result'
    arguments: pWinner,pLoser

    onCombatCalc
    'Combat Result'
    arguments: cdAttacker, cdDefender, iDefenderOdds

    onCombatHit
    'Combat Message'
    cdAttacker, cdDefender, iIsAttacker, iDamage

    onImprovementBuilt
    'Improvement Built'
    arguments: iImprovement, iX, iY

    onRouteBuilt
    'Route Built' (note: routes = road or railroad)
    arguments: iRoute, iX, iY

    onPlotRevealed
    'Plot Revealed'
    arguments: pPlot, iTeam

    onBuildingBuilt
    'Building Completed' (note: building = improvement or wonder)
    arguments: pCity, iBuildingType

    onProjectBuilt
    'Project Completed' (note: project = space ship part, Appollo Program, 1 or 2 other modern wonders)
    arguments: pCity, iProjectType

    onUnitMove
    'unit move'
    arguments: pPlot,pUnit

    onUnitSetXY
    'units xy coords set manually' (note: i.e. teleport)
    arguments: pPlot,pUnit

    onUnitCreated
    'Unit Completed'
    arguments: unit

    onUnitBuilt
    'Unit Completed'
    arguments: city, unit

    onUnitKilled
    'Unit Killed'
    arguments: unit, iAttacker

    onUnitLost
    'Unit Lost'
    arguments: unit

    onUnitPromoted
    'Unit Promoted'
    arguments: pUnit, iPromotion

    onUnitSelected
    'Unit Selected'
    arguments: unit

    onUnitRename
    'Unit is renamed'
    arguments: pUnit

    onGoodyReceived
    'Goody received'
    arguments: iPlayer, pPlot, pUnit, iGoodyType

    onGreatPersonBorn
    'Unit Promoted' (note: obviously not, should be 'Great Peson Born')
    arguments: pUnit, iPlayer, pCity

    onTechAcquired
    'Tech Acquired'
    arguments: iTechType, iTeam, iPlayer, bAnnounce

    onTechSelected
    'Tech Selected'
    arguments: iTechType, iPlayer

    onReligionFounded
    'Religion Founded'
    arguments: iReligion, iFounder

    onReligionSpread
    'Religion Has Spread to a City'
    arguments: iReligion, iOwner, pSpreadCity

    onGoldenAge
    'Golden Age'
    arguments: iPlayer

    onEndGoldenAge
    'End Golden Age'
    arguments: iPlayer

    onChangeWar
    'War Status Changes'
    arguments: bIsWar, iPlayer, iRivalTeam

    onChat
    'Chat Message Event'
    arguments: chatMessage

    onSetPlayerAlive
    'Set Player Alive Event' (note: either when a player dies or is revived (only in scenarios))
    arguments: iPlayerID, bNewValue

    onCityBuilt
    'City Built'
    arguments: city

    onCityRazed
    'City Razed'
    arguments: city, iPlayer

    onCityAcquired
    'City Acquired'
    arguments: owner,playerType,city,bConquest,bTrade

    onCityLost
    'City Lost'
    arguments: city

    onCultureExpansion
    'City Culture Expansion'
    arguments: pCity, iPlayer

    onCityGrowth
    'City Population Growth'
    arguments: pCity, iPlayer

    onCityDoTurn
    'City Production'
    arguments: pCity, iPlayer

    onCityBuildingUnit
    'City begins building a unit'
    arguments: pCity, iUnitType

    onCityBuildingBuilding
    'City begins building a Building'
    arguments: pCity, iBuildingType

    onCityRename
    'City is renamed'
    arguments: pCity

    onVictory
    'Victory'
    arguments: iTeam, iVictory

    onGameUpdate
    'sample generic event, called on each game turn slice' (note: at least in theory about 4 times per second (didn't test it))
    arguments: turnSlice

    onMouseEvent
    'mouse handler - returns 1 if the event was consumed'

    arguments: eventType,mx,my,px,py,interfaceConsumed,screens

    Example
    Finally, I'll give one simple example of how to use all this code in practice. For more examples, see the Python code in the Mods folder, the various scenarios come with a lot of interesting code.

    Code:
    def onGameStart(self, argsList):
    	'Create a popup message at the start of the game'
    	popup = PyPopup.PyPopup()
    	popup.setBodyString( 'Hello World' )
    	popup.launch()
    
    def onCityBuilt(self, argsList):
    	'For player 1, create a Warrior (index 17) in every city that is built'
    	city = argsList[0]
    	if city.getOwner() == 1:
    		city.getOwner().initUnit(17, city.getX(), city.getY(), UnitAITypes.NO_UNITAI)
    Note that this example overwrites the existing event code, normally you will want to either append to it, or subclass it (more info on that will follow), for simplicity's sake I didn't do that here.

    If anyone has any Python-related questions, I'll try to answer them as best as I can.
    Last edited by Locutus; October 26, 2005, 16:28.
    Administrator of WePlayCiv -- Civ5 Info Centre | Forum | Gallery

  • #2
    Nice one, Locutus

    Comment


    • #3
      excellent, it looks like some stuff can be converted from SLIC (kind of)/

      Is there a way to select unit (or building) traits. like if unit has move of 2 than bonus attack etc?
      Formerly known as "E" on Apolyton

      See me at Civfanatics.com

      Comment


      • #4
        Nice dude. Thanks, etc.

        I'm amazed to see onUpdate and onGameUpdate. Does this mean a scripter can play with the graphics engine? If so, then together with the mouse and keyboard events it seems like you could script a simple arcade game right into Civ4. O_o.

        Also wondering the extent of UI modification possible via scripting. Can new widgets/screens be added? Can other UI elements be disabled?

        I saw in a screenshot that that scripts could be attached to a city. When is this attached script called by the program? Can a script be attached to other objects---units, specific units, techs, plots, civilizations etc?

        (...taps foot and waits for copy of Civ4 to arrive on Friday...)

        Comment


        • #5
          Good.
          The Sherrin Foundation
          Captain of the Concordian Armed Forces, Inspectorate of the MoD Term VI

          Comment


          • #6
            Originally posted by E
            excellent, it looks like some stuff can be converted from SLIC (kind of)/
            Yeah, I made sure of that At least in theory almost all SLIC scripts should be convertable to Civ4 (didn't actually try though). The only exception might be in some of the pre events, it was decided that those would be too bug-prone and too time consuming to get right. But when the SDK is released we can add these ourselves of course

            Is there a way to select unit (or building) traits. like if unit has move of 2 than bonus attack etc?
            Yes, everything that subclasses from CvInfoBase takes care of that. CvUnitInfo().getMoves() returns the movement points of a unit. To get the movement points of a Warrior:

            Code:
            gc.getUnitInfo(gc.getInfoTypeForString("UNIT_WARRIOR")).getMoves()
            Note that 'gc' is shorthand for CyGlobalContext(), which is defined at the start of pretty much every Python file because it's so common.

            gc.getInfoTypeForString("UNIT_WARRIOR") will return 17, the index of the Warrior unit, as you could see in the example in my opening post. The difference between this and using 17 directly is that this requires less cross-referencing with the XML files for the programmer and is less prone to errors when you mod the XML files. The downside is that it's slower (though your code has to make a LOT of calls to it before you start to notice the difference).

            So apart from the syntax it works pretty much the same as the Database functions in CtP2.

            Originally posted by drekmonger
            I'm amazed to see onUpdate and onGameUpdate. Does this mean a scripter can play with the graphics engine?
            No. It just means those events are triggered every frame or every game slice. This is mostly useful if an existing event doesn't do the job and you want to check for a specific thing that might happen in the game. Of course, since this code gets called *very* often, best not to use it too much and keep it as efficient as possible, or you could notice a real drop in performance.

            But yeah, the code is pretty powerful, even without the SDK you could probably make an arcade mini-game in Civ4 if you wanted

            Also wondering the extent of UI modification possible via scripting. Can new widgets/screens be added? Can other UI elements be disabled?
            Yes, you can add and remove pretty much anything you want. A few things are hardcoded, but the degree to and the ease with which the UI can be modded in Python is the thing that surprised me most (in a good way) when I got into Civ4 modding. In the Desert War scenario that comes with the game I showed off some of the things you can do with it, though it only barely scratches the surface of what is possible.
            Administrator of WePlayCiv -- Civ5 Info Centre | Forum | Gallery

            Comment


            • #7
              Originally posted by drekmonger
              I saw in a screenshot that that scripts could be attached to a city. When is this attached script called by the program? Can a script be attached to other objects---units, specific units, techs, plots, civilizations etc?
              Don't DanS me

              That was added pretty late in the development so I haven't really messed with that yet, but I think it's just so you can add scriptdata to objects. Just doing that in the World Builder isn't very useful though, as far as I can see -- but maybe Trip or whoever added that to the World Builder had something in mind I haven't thought of. Script data is really only useful if you have a script to handle it. It's basically just useful to associate specific data with a specific unit/city/tile/whatever -- and to make scripts savegame-safe, but I'll discuss that in my upcoming full Python guide. As it stands, in my current (non-final) build, it doesn't work anyway, just gives me an error (or maybe I'm using it the wrong way).
              Administrator of WePlayCiv -- Civ5 Info Centre | Forum | Gallery

              Comment


              • #8
                I'm glad to see you had a strong hand in Civ4...you are making the ctp2 to civ4 leap much more tempting...

                Another question though. would it be possible in Python (or do we have to wait til the SDK) to add new unit flags? like in my ctp2 code I added cultureonly and citysyleonly flags. is it possible to just create a flag in the xml and call it through python (assuming the game would ignore the reference). i'm sure its a long shot but just hoping

                offtopic. I have another thread asking about graphic formats are you able to answer those questions now, especially can we add new citystyles, or what format leaderheads are in.

                thanks Loc
                Formerly known as "E" on Apolyton

                See me at Civfanatics.com

                Comment


                • #9
                  Originally posted by E
                  I'm glad to see you had a strong hand in Civ4...you are making the ctp2 to civ4 leap much more tempting...
                  It's all part of Soren's nefarious plan to conquer the world (or the Civ community anyway)!!

                  Another question though. would it be possible in Python (or do we have to wait til the SDK) to add new unit flags? like in my ctp2 code I added cultureonly and citysyleonly flags. is it possible to just create a flag in the xml and call it through python (assuming the game would ignore the reference). i'm sure its a long shot but just hoping
                  That's probably not possible with Python-only because as you can see in my example, to check for an XML property, you have to call a specific API function. There's no way to generically search for properties, as there is for main entries such as unit/civic/building/etc types, as far as I know (but even I am still finding new applications for Python every now and then). E.g. to check the XML unit property iMoves, you have to use the Python call getMoves(). If you were to define your own bCultureOnly flag, you would need to add a getCultureOnly call in the game engine to be able to use it in Python. You'll need the SDK for that, and even then I don't dare say with 100% certainty that it's possible (90% sure it is though).

                  Either way, you *can* just define a list of all unit types (or city styles or civics or whatever you need) in Python that should have a certain property, and give them a specific value if needed. For example, something along the lines of this:

                  Code:
                  iNumSlavesDict = {
                  	"UNIT_WARRIOR" : 1,
                  	"UNIT_SETTLER" : 2,
                  	"UNIT_CATAPULT" : 0,
                  	# [...]
                  	"UNIT_TANK" : 1
                  }
                  # [...]
                  def onUnitKilled(self, argsList):
                  	unit, iAttacker = argsList
                  	strUnitType = gc.getUnitInfo(unit).getType()
                  	if strUnitType in iNumSlavesDict.keys():
                  		iNumSlaves = iNumSlavesDict[strUnitType]
                  		self.addSlaves(iAttacker, iNumSlaves)
                  # [...]
                  def addSlaves(self, iPlayer, iNumSlaves):
                  	# some implementation to add slaves to a player
                  One of the major advantages of Python over SLIC is that it's a much more powerful language that among other things supports much more powerful datatypes, as you can see here, which makes adding completely new features, including new unit flags, much easier to do.

                  offtopic. I have another thread asking about graphic formats are you able to answer those questions now, especially can we add new citystyles, or what format leaderheads are in.
                  As you should know by now, if there's one thing I totally suck at it's graphics. So I'm not really the person to answer those types of questions. I'll have a look at your thread but I can't promise any answers.
                  Last edited by Locutus; October 26, 2005, 17:18.
                  Administrator of WePlayCiv -- Civ5 Info Centre | Forum | Gallery

                  Comment


                  • #10
                    E.g. to check the XML unit property iMoves, you have to use the Python call getMoves(). If you were to define your own bCultureOnly flag, you would need to add a getCultureOnly call in the game engine to be able to use it in Python. You'll need the SDK for that, and even then I don't dare say with 100% certainty that it's possible (90% sure it is though).
                    Do you have details on how the xml and python interact? I guess there are several xml files. Is there some description of what they are somewhere? From what you say it looks like Python can be used to ask info from the xml files, rather than the xml used to call some python code.

                    I don't know Python (yet), but in java it is possible to find out which methods an object sports using reflection. I'd be surprised if Python didn't have a functionality like that. So if the xml is used to call back python methods, in theory, if you subclass the python objects, you could add methods on them and call them from the xml? I know in theory this is possible (the xml parser we use in Clash is based on that principle: find an object, use reflection to find the method with the name matching that in the xml file, and call it) but it depends whether the xml files drive the game (as n Clash) or the scripts read from the xml files (which would be much weaker).
                    Clash of Civilization team member
                    (a civ-like game whose goal is low micromanagement and good AI)
                    web site http://clash.apolyton.net/frame/index.shtml and forum here on apolyton)

                    Comment


                    • #11
                      Originally posted by LDiCesare
                      Do you have details on how the xml and python interact?
                      As you mention yourself, I gave an example of how to do it. You can use CvInfoBase and subclasses to extract info from the XML files.

                      I guess there are several xml files. Is there some description of what they are somewhere?
                      They are pretty self-explanatory. I haven't seen any documentation for them but it usually doesn't take me more than a few moments to find find what I need. There are a few files with global settings and there is a bunch of folders that sort the XML files by topic: the XML/Units folder contains all unit, promotion and command related XML files, XML/Civilizations everything related to civs and leaders, XML/Art contains all the art definitions, XML/Text all the in-game text (in all supported languages), etc.

                      As an example, XML/Civilizations contains 4 files: CIV4CivilizationInfos.xml (with things like civ name/adjective/etc, city names, art styles, UUs, starting techs, etc), CIV4CivilizationsSchema.xml (containing the Schema info for the files in this folder), CIV4LeaderHeadInfos.xml (with info on leaders and their personality) and finally the small CIV4TraitInfos.xml (containg info on trait properties). The setup for the other folders is similar. Sometimes there's only 2 files, sometimes there are close to close to 20. And then there are GameInfo and Misc which contain random stuff that doesn't warrant its own folder for some reason (game options, difficulty levels, civics, rivers, tutorial info).

                      From what you say it looks like Python can be used to ask info from the xml files, rather than the xml used to call some python code.
                      There are a small number of hooks to Python in XML as well but I haven't used them yet, not sure how to do it. Not sure they're very interesting for modmakers either, seems mostly 'under-the-hood' stuff that's needed to run the game. Though I'm sure someone will eventually find some kind of use for them

                      I don't know Python (yet), but in java it is possible to find out which methods an object sports using reflection.
                      Reflection is actually one of the key strengths of Python as a language, it probably does a better job than Java in this respect (though I never really used that aspect very extensively in either language, so don't take my word for it). There's a whole chapter dedicated to in this guide, which incidentally I would recommend as a good source for learning Python if you're an experienced programmer and want to learn all the ins and outs of the language. (If you're only interested in the parts that are relevant for Civ4, wait for my guide.)

                      I'd be surprised if Python didn't have a functionality like that. So if the xml is used to call back python methods, in theory, if you subclass the python objects, you could add methods on them and call them from the xml?
                      I don't *think* so, but as I said, I haven't really used the Python hooks in XML yet, so I'm not an expert. The game is definitely not driven on XML though, it's just the format in which the database values are read during load, pretty much exactly like the TXT files in CtP (anyone who's modded CtP1/2 will notice that those games were a major inspiration for the modding aspects of Civ4).

                      I'm not sure if that makes it weaker, as I see it it just means that Python and the DLL (SDK) drive the XML rather than the other way around, the net result would seem the same to me. But then again, I'd really have to know more about the Clash approach, never really dug too deep into such fundamental issues (not like I can change it -- I had a big say in the development of the modding aspect of the game but not that big a say ) What kind of new functionality are opened up by it?
                      Last edited by Locutus; October 26, 2005, 19:05.
                      Administrator of WePlayCiv -- Civ5 Info Centre | Forum | Gallery

                      Comment


                      • #12
                        The aspect of modding in clash opened by the use of reflection is that anything accesssible by the java language is actually accessible from the xml. We (over?)use xml so events are also coded with that rather than Python in CIV. Since CIV uses a scripting language, (doesn't require compilation - comes with lots of features), using the same kind of approach would allow to add for instance new data fields to units in some Python code, value them in the xml files, and exploit these in some python. I'll have to wait a bit to see if that's possible, and I don't know if that's the way to add data to units (adding homemade promotions might achieve the same?). So you could add stuff like, f.e. a weapon slot, and use this as a base for a unit workshop factory for instance (add data to unit - use it to alter its abilities).
                        Clash of Civilization team member
                        (a civ-like game whose goal is low micromanagement and good AI)
                        web site http://clash.apolyton.net/frame/index.shtml and forum here on apolyton)

                        Comment


                        • #13
                          At this point, scenario creation looks like a daunting task (unlike Civ II) for us non-programmer types. confused:

                          Comment


                          • #14
                            donbrid:

                            If you just want to create a scenario using the existing rules in Civ4, I would recommend looking through my manual on the World Builder. You can create a full scenario based on the Civ4 rules very easily. No need to look into xml or python to make a good scenario.

                            Dale

                            Comment


                            • #15
                              (Dale beat me to it as I was composing this; but you're going to get it anyway.)

                              Well, it depends on what kind of scenarios you want to make. Did you do anything with the Civ2 events 'language'?

                              I don't remember a lot of the Civ2 scenarios I used to play as having a lot of scripting. But this was in the last millenium and I'm sure that the ones that have come out more recently are way more sophisticated.

                              Anyway, this thread is about scripting with Python but if you want to make a basic scenario (like those early Civ2 scenarios) you don't need to worry about it. You can start by reading Dale's Civ IV Modding Manual (it's stickied at the top of this forum). That gives you lots of info about how to use the World Builder.

                              The other thing you'll want to edit is the XML files. But this is in general surprisingly easy: they may look daunting at first, but they're actually very easy to play with. They're just a *very* big expansion of those Civ2 txt files you used to edit.

                              And if you come up with a good looking, well designed scenario, you just might be able to persuade someone to do some scripting for you.

                              Comment

                              Working...
                              X