Introduction to Fuzzy 
Symbolic Connectionist 
Network

(c) 1995, Universal Problem 
Solvers



Overview of fSC-Net

fSC-Net is a hybrid symbolic/connectionist network that utilizes 
fuzzy logic as its mean to perform uncertainty management.  The 
main purpose of fSC-Net is to act as a knowledge acquisition tool, 
which can be used by domain experts in the development of expert 
systems.  The main learning algorithm is centered around a 
mechanism that supports the automatic construction of the 
network topology.  The connectionist network encodes the learned 
knowledge.  By incorporating symbolic structures into the network 
itself, it is possible to represent both structured as well as 
unstructured variables and rules.  Rules can not only be added as 
domain specific knowledge, but can also be extracted after 
learning or refined during learning.  Finally, fSC-Net supports the 
direct incorporation of fuzzy variable membership functions 
through the user, or by automatically learning the appropriate 
membership functions for any given input



Why Opt for a Hybrid 
Representation?


By combining both symbolism and connectionism an attempt is 
made to harness the virtues of both representations by minimizing 
their individual shortcomings. Let us next explore those aspects 
which favor each of the two seemingly dichotomous 
representations



Symbolic Representation


  The following exemplifies some of the most important aspects 
for choosing symbolism to represent knowledge:

 Knowledge refinement by means of rule encoding.
  Consultation and explanation facilities commonly found in 
expert systems (rule extraction)

Symbolic structures such as variables, comparators, quantifiers, 
etc. are more readily understood and accepted by humans than, for 
example, hyperplanes and weight vectors



Connectionist Representation

  Three of the most important features common to connectionism 
are:

 Allows for a highly parallel and uniform representation of 
knowledge.
  Possible fault and noise tolerance.
 Readily deals with continuous inputs and outputs



fSC-Net Networks


Since fSC-Net favors a number of different symbolic structures in 
its representation of knowledge, its networks tend to be 
significantly larger than traditional connectionist networks. On the 
other hand, fSC-Net generated networks are sparsely connected 
and tend to stretch over several layers. In the next section, an 
overview of the basic unit activation functions utilized by fSC-Net 
isprovided



Types of Unit 
Activation Functions


  fSC-Net incorporates five different types of network units, each 
dedicated to assume a specific purpose.  
  Legend:

 Current output of unit
 New activation of unit.
 Bias of unit.
 Weightofconnection.



Min and Max 
Activation Functions


The min and max transfer functions simply calculate the activation 
of a network unit as the minimum or maximum of the product of 
incoming activation and connection weight, respectively:
 MinActivation



Other Activation Functions



  The remaining activation functions utilized by fSC-Net are:

 Linear Activation (i.e., LT):
  
  Output:
  
 Negation



Unit Transfer Function



The final output of every fSC-Net network unit is calculated by 
passing the newly calculated unit activation through the following 
transferfunction



Incremental Learning



fSC-Net utilizes an incremental learning algorithm to construct the 
basic knowledge base from a collection of training patterns.  In 
other words, only a single pass over the training patterns is 
required.  This allows for rapid prototyping of expert networks.  
Adding new knowledge (i.e., training patterns) is as easy as 
loading an existing expert network and learning the new patterns 
on top of the already existing network.  As pointed out earlier, 
fSC-Net learns new patterns by automatically constructing an 
appropriate network topology based on the training patterns seen 
over time (i.e., order effects training outcomes).  The incremental 
training algorithm utilized by fSC-Net is based on the following 
decisionprocess:



Recruitment of Units


fSC-Net uses a simple set of tests to decide how a new training 
pattern is incorporated in the network that is currently under 
construction:
  
If the actual and the expected output are similar within some error 
e no action is taken (the pattern is considered learned).

If the actual and expected output differ by no more than 5e, the 
bias of earlier created network units is appropriately modified to 
reflect the change introduced by the new pattern.

If the difference is larger than 5e a new set of hidden units is 
created to encode the training pattern (the pattern is considered 
dissimilar  to any previously stored patterns)



Output Unit Network 
Structure



Since fSC-Net supports uncertainty management, the user can 
assign degrees of uncertainty to individual training inputs.  To 
guarantee that uncertain information is correctly combined, fSC-
Net collects both positive as well as negative evidence.  
Consequently, both pieces of evidence need to be combined to 
create a final composite.  This is achieved by setting aside for 
every system output a network structure, which consists of 4 
hidden network units:
  

 The final output unit.
  A positive collector unit (i.e., PC collects positive evidence).
 A negative collector unit (i.e., NC collects negative evidence).
 The unknown unit (i.e., UK always propagates an activation of 
0.5, thereby indicating unknown).
   
Note, an activation of 1.0 indicates complete presence, whereas 
0.0 denotes completes absence. The next page displays an example 
of this output structure



Example of Output 
Structure



  The numbers located within the network units indicate the bias of 
the unit



Types of Variables



fSC-Net supports two basic types of variables: structured and 
structureless.  Structureless variables are of the simplest form and 
require the least amount of hidden unit representation.  In general, 
a structureless variable is a symbolic input that has an uncertainty 
measure associated with it.  


  Example:

  X 0.8 

  Here, X designates an input with an uncertainty measure of 0.8.
  Structured variables on the other hand can come in two flavors:
 Fuzzy
 Nomina



Fuzzy Variables



Fuzzy variables allow the user of the system to divide (i.e., 
partition) the numerical range of a variable into its fuzzy 
equivalent.  In general, fuzzy variables are described by a set of 
membership functions, where each such function is associated 
with a linguistic hedge (i.e., high, small, large, etc.).  The 
membership functions correlate a given numerical value with a 
degree of membership indicating the strength of the numerical 
value being a member of the predefined fuzzy sets. A linguistic 
hedge within fSC-Net is described by four quantities: 
  
  Linguistic Hedge  = <Lower Bound, Upper Bound, Lower 
Plateau, Upper Plateau>.
  
Whenever the value of a given fuzzy variable lies within the range 
defined by the two quantities Lower Bound and Upper Bound, the 
membership value of the Linguistic Hedge is defined as 1 (i.e., 
complete presence)



Fuzzy Variables Continued



If on the other hand the value of a fuzzy variable falls outside the 
interval spanned by the two quantities Upper Plateau and Lower 
Plateau, a membership value of 0 (i.e., complete absence) is 
assigned.  For all remaining cases, a graded response in 
membership value is provided in the form of a linear function.  
The following figure displays an example of a single membership 
function (p-shaped membership function)




Fuzzy Linguistic Hedge 
Network



The following represents an example of the type of subnetwork 
fSC-Net creates for representing a fuzzy linguistic hedge.  In this 
particular case the fuzzy variable is Age and the linguistic hedge 
implemented by the network is Middle-Aged



Fuzzy Definition Files



Fuzzy variables are defined in fSC-Net within a special definition 
file (i.e., file extension is *.var).  Below follows an example of a 
single definition of a fuzzy variable Age and three linguistic 
hedges (i.e., Child, Teenager, and Old).  A typical fSC-Net 
variable file can contain a number of these definitions



  
  
Automatic Generation of Fuzzy 
Variable Definition Files


If the user does not supply a fuzzy variable definition file 
(extension *.var) prior to training a new network, fSC-Net will 
automatically create this file.  It utilizes a special partitioning 
algorithm that attempts to anticipate the number of linguistic 
hedges required for learning some task.  Note, that fSC-Net may 
generate more linguistic hedges than required during learning.
Once a fuzzy variable definition file has been loaded, and new 
training patterns have been supplied, fSC-Net will automatically 
modify the membership arms of the individual linguistic hedges.  
The user can of course extract the newly formed membership 
functions.



Nominal Variables



Besides allowing fuzzy variables fSC-Net also supports variables 
with nominal inputs (i.e., inputs that can be assigned a finite 
number of symbolic values).  At any one time only a single value 
can be assigned to a variable.  Associated with every value 
assignment is a belief value indicating how certain it is that the 
assigned value is the right one.   In fSC-Net it is not necessary to 
predefine all nominal variables and their possible values.  Creating 
the appropriate internal neural subnetworks for encoding this 
symbolic knowledge is automatically handled by the system.  This 
gives it unprecedented flexibility since the process of variable 
definition is completely transparent to the user.  The next page 
contains an example of the type of subnetwork generated by fSC-
Net for the variable Color and its three values Red, Blue, and 
Green



Example of Nominal 
Variable



Pruning Networks



fSC-Net provides a mechanism for pruning connections from 
created networks.  The pruning procedure can be invoked once 
learning has completed.  Invoking the pruning mechanism can 
under certain conditions result in long execution times (i.e., 
increase in iterations) and should therefore be carefully used



Training File Format



The format for training files(extension *.trn) is quite simple.  The 
first line of every file contains the string NTrainingPatterns and 
the number of training examples that should be read by the system.  
Next, each training pattern is listed.  This pattern consists of two 
parts: input list and outputs.  Inputs can be preceded by the 
keywords Fuzzy or Nominal, if they represent structured variables, 
or no keyword at all (unstructured format).  Next, the inputs  
identifier is stated followed by its value and the degree of 
certainty.  Once all inputs have been listed, the keyword Outputs is 
used to separate the list of inputs from the list of outputs.  Every 
output entry consists of the output identifier and the output value.  
The output value can either be a degree of certainty (if the data set 
contains classificatory data) or a scaled continuous value (for all 
continuous data).  The user needs to scale all values unto the unit 
interval (i.e., [0,1]).  Note, the user is also required to select prior 
to training of a new network, whether the training data represents 
classificatory data (i.e., discreet outputs with an associated 
uncertainty) or whether the outputs are continuous.  The selection 
can be made from within the parameter dialog box located under 
View of the Parameter menu option (.i.e., select Continuous 
outputs on ).  By default data is considered to be classificatory.  
Also note, that the format for test files is the same as for training 
files.  If a value does  not exist for an output, it is simply not listed.  
The next page outlines an example of a training file



Example Training File


  The following is an example of a training file:
  
  NTrainingPatterns 2
  
  Fuzzy		Age	25.5	0.85
  Nominal	Color	Red	0.70
  PayRaise	1.0
  Outputs
  Successful	0.80
  End
  
  Fuzzy		Age	39.0	0.96
  Nominal	Color	White	0.8
  PayRaise	0.65
  Outputs
  Successful	0.59
  End
  



Incomplete Training Data



In many instances not all training inputs or outputs may be known.  
To deal with missing data in fSC-Net is simple. If a parameters 
value is not available, or it is considered unimportant, do not list 
it.  fSC-Net applies a policy of out of sight out of mind for dealing 
with missing or redundant information.  In fact, it is beneficial to 
cut back on network growth by eliminating as much information 
as possible.  For example, if you are dealing with mutually 
exclusive outputs (i.e., only one output can be active at any one 
time), it is best to only list the output which is considered 
activated in the output list of the training patterns.  It is indeed 
wasteful to list the fact that an output is not present!!



Rules in fSC-Net



fSC-Net allows users to enter domain knowledge in the form of 
rules.  The user can then choose, if he/she would like to refine 
these rules, by learning new information on top of the already 
existing rule set.  At any point after training, the user can extract 
rules from the created neural network for inspection.  Rules are 
represented in operator prefix notation.  The next page provides a 
simple example of a rule file



Example Rule File


  The following is an example of a rule file:
  
  Rule 1	if and(Nominal(Color[Red]) = 0.95,
            	           Fuzzy(Age[Young]) = 1.0,
           	           PayRaise = 0.85) then Successful (0.86);
  
  Rule 2	if and(Nominal(Color[White] )= 1.0,
      	          PayRaise = 1.0) then Successful (1.0);
  
  Rule 3	if or(Nominal(Color[White] )= 1.0,
      	         PayRaise = 0.9) then Successful (0.84);
  


  
  
Getting Started


The purpose of this section is to give the user a walk through of 
some of the more important features of fSC-Net.  The example 
used throughout this section describes the output behavior of a 
simple DC motor.    
Lets start off, with creating a network  and then proceed with 
training it. From the Network menu option select Create.  You 
need to do this every time you want to create a new network!  Now 
set the parameter Continuous Outputs On.  You will find it located 
in the Parameter View Dialog Box.  To select it go to the 
Parameter menu option and select View.  Next, select File from the 
Train menu option.  The standard file dialog box will appear.  Go 
to the directory domains.  Once you have located the directory 
select the file dc-motor.trn.  When prompted to enter an output file 
(will contain a log of the training process) simply enter dc-
motor.out.  Once you hit the OK button, training will commence.  
Once it is completed, a message box will appear informing you, 
for example, of how long training lasted.  To display the 
constructed network, select Display from the Network menu 
option. 



Getting Started Continued




If you would like to find out more information for each of the 
displayed network units, simply move the cursor to the desired 
unit and press the left mouse button.  A message box will appear 
providing several important pieces of information.  For all units 
which are not inputs or outputs, you will be asked if you would 
like to see the units connections and weights.  If you select Yes, a 
new dialog box will appear, showing you all incoming and 
outgoing connections and their associated weights.  The 
connections are identified by a unique unit name.  Note, that this 
feature is not available for output units, since they always have 
two inputs originating from their associated Positive (i.e., PC) and 
Negative (i.e., NC) collector  units.



Getting Started Continued




To test how well the network is performing, you simply select File 
from under the Test menu option.  When asked to enter a file of 
test patterns select the file dc-motor.trn.  Note, that in fSC-Net 
train and test files have the same format.  For the output file enter 
the file dc-motor.tre.  Once you select the OK button, testing will 
commence.  During this phase, the displayed network units will 
change their color reflecting the degree of activation.  Recall, that 
all unit outputs are scaled to the unit interval.  Hence, the closer 
the value is to 1 the more red the color of a unit appears, whereas a 
value close to 0 will result in a deep blue color.  A value of 0.5 
(i.e., unknown) will result in a deep black color selection



Getting Started Continued



Once testing is completed, a message box will appear informing 
you of how well the network has responded to the tested patterns.  
This of course implies that the parameter Accuracy (located within 
the Parameter View Dialog Box) is selected.  If it is not, no 
accuracy is calculated.  You would only choose this option, if you 
intended to use the test facilities of fSC-Net to classify new 
patterns. One of the error measures employed by fSC-Net is 
Average Prognostic Error.  It is calculated as follows:



Getting Started Continued



Once you have tested the network, you can extract rules from it.  
Simply select Extract from the Rule menu option and enter the file 
name dc-motor.rul.  A message box will appear and inform you of 
how many rules were extracted.  You can view these rules by 
selecting Edit from the File menu option.  Once the standard file 
dialog box appears, use the List Files of Type box to select file 
extension *.rul.  Next, open the the file dc-motor.rul.  The edit box 
will popup and display the desired file.  Note, this box can hold 
files of up to 2^15 characters.  You can also use it to make 
changes to files (maybe edit a set of rules and then reload them).  
To exit the edit box, just move the cursor out of the edit box and 
press the right mouse button.
  This completes the getting started section. 



File Menu Option



Edit Option: Brings up a simple edit box, which can be utilized for 
displaying and editing small files (max 2^15 characters).  You exit 
the edit box by moving the cursor outside the edit box and 
pressing the right mouse button.
Net to Trn Option: Allows the user to translate a file of training 
patterns from the Net format (extension *.net) to the fSC-Net Trn 
format (extension *.trn).  Note, it is assumed that all outputs are 
mutually exclusive and discreet.  The Net format is used by two 
other applications which are available from Universal Problem 
Solvers (i.e., Trans-Dimensional Learning and Multi-Pass 
Instance-Based Learning).
 Exit Option: Terminates fSC-Net



Network Menu Option



 Create Option: Used to create a new network.  Needs to be 
selected before a new network is trained.
  Load Option: Allows the user to load a saved network (extension 
*.sav).
 Save Option: Used to save a created network unto disk.
Display Option: Allows the user to display a created network 
graphically.  To obtain information on specific network units 
simply point the cursor arrow at the unit and press the left mouse 
button.
 Resources Option: Provides information with regard to network 
resources available e.g.., connections and units



Variables Menu Option



 Create Option: Creates a new network.  This command needs to 
be executed prior to learning a new data set.
Extract Option: Allows the user to extract all currently loaded 
fuzzy variables and their linguistic hedges.  The output file will be 
stored with extension *.var.
View Option: Provides a tool for displaying the membership 
function of a linguistic hedge.  The user is prompted to enter a 
fuzzy variable definition file.  Once a file has been selected, a 
dialog box with two list boxes will popup.  The left box will 
contain a list of all fuzzy variables.  By double clicking on any 
given fuzzy variable, its associated linguistic hedges are loaded 
into the right box.  The user can select any of these hedges and by 
pressing the Show button graphically display it



Train Menu Option


Train Option: Allows the user to enter a file of training patterns 
using the standard file location dialog box.  Training files have 
extension *.trn.
Double Bar: Whenever a new data set is loaded and trained, fSC-
Net collects statistical information, such as training time.  The user 
can recall this information by selecting the appropriate file 
identifier which will be automatically placed beneath the double 
bar.  Up to 12 such statistics can be collected at any one time



Test Menu Option



Test Option: Allows the user to enter a file of  test patterns using 
the standard file location dialog box.  Test files have the same 
extension *.trn, as training files.
Double Bar: Whenever a new data set is loaded and tested, fSC-
Net collects statistical information, such as training time.  The user 
can recall this information by selecting the appropriate file 
identifier which will be automatically placed beneath the double 
bar.  Up to 12 such statistics can be collected at any one time



Rules Menu Option


Extract Option: Once an fSC-Net Network has been created and 
trained, the user has the option to extract the knowledge contained 
in the network in the form of standard  rules.  These rules are in 
operator prefix notation. Note, the definition file for fuzzy 
variables is not automatically created.  
Load Option: Using this option the user can load in rules.  The 
format is the same as the one used in the Extract Option.  By 
utilizing both options in conjunction with the train option, the user 
can engage in knowledge refinement



Parameters Menu Option



View Option: Contains several parameters that the user can 
modify which influence the types of data sets handled, the degree 
of abstraction utilized by fSC-Net and several other parameters.
Option Prune: Provides a means for the user to prune excess 
connections from an fSC-Net network.  The process can be time 
consuming.  It is most effective for data sets which contain mostly 
discreet inputs and outputs



View Option Parameters




Min Drop Parameters: These parameters allow the user to control 
the degree of generalization displayed by fSC-Net.  If the Min 
Drop option is not selected, an fSC-Net network will respond to 
new patterns much like any other neural network.  If  a new pattern 
can not be matched (i.e., recognized) none of the outputs will be 
turned on.  The user can utilize the MinDrop Option to force fSC-
Net to generalize in such a case.  When Min Drop is activated 
fSC-Net simply keeps dropping (i.e., ignores) pieces of evidence 
which display the highest degree of uncertainty (i.e., have certainty 
values closest to 0.5, unknown).  Once an outputs certainty 
exceeds a predefined threshold (defined by Upper Threshold 
parameter) generalization stops.  At most Maximum (parameter 
that can be set by the user) pieces of evidence can be dropped (i.e., 
ignored)



View Option Learning 
Parameters


 Learning Parameters: These two parameters control how many 
training patterns fSC-Net remembers (i.e., stores) versus the 
amount of abstraction it uses.  Of these two parameters Epsilon 
is the more important.

Epsilon: This parameter is equivalent to the one discussed in a 
previous section on learning in fSC-Net.

Alpha: Determines how much impact a new pattern has on an 
already stored pattern (i.e., the bias of the network unit that 
encodes said pattern).  A high alpha value indicates that a new 
pattern has less influence on changing the bias of a unit 
representing a stored pattern.  The range of alpha is [1,MAX], 
where MAX is the largest representable integer



View Option Other 
Parameters


Accuracy On: If this parameter is set fSC-Net will calculate the 
performance accuracy of a network when it is tested against a data 
set.  The parameter is turned off, if the user wants to perform 
classification of data patterns.
Cont. Output On: By setting this parameter the user indicates that 
the training data contains continuous outputs.  If it is not set the 
data is assumed to contain only discreet outputs with an associated 
certainty factor.
 PSA On: Allows fSC-Net to use a pre-selection scheme of input 
features before pruning is initiated.
 Xclusive Outputs: Indicates that all the outputs of a training set 
are considered mutually exclusive.
 Test Graphics On: If this parameter is set testing will be shown in 
graphical mode. 



Help Menu Option



 Memory Resources: Indicates how much global memory and 
stack space are available for the users system.
  About fSC-Net: Displays version number of this system.
